[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127579900523-11321-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:33:00 dist-test-slave-6k22 krb5kdc[11328](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:33:00 dist-test-slave-6k22 krb5kdc[11328](info): set up 2 sockets
Aug 07 18:33:00 dist-test-slave-6k22 krb5kdc[11328](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260807 18:33:01.094575 11321 mini_kdc.cc:121] Time spent starting KDC: real 1.124s user 0.000s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127579900523-11321-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127579900523-11321-0/krb5kdc/test-user.keytab.
Aug 07 18:33:01 dist-test-slave-6k22 krb5kdc[11328](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127581, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:33:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:33:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.14.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.14.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.14.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.126.keytab.
Entry for principal kudu/127.11.14.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.126.keytab.
I20260807 18:33:01.236152 11321 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkz56XN/build/debug/bin/kudu
/tmp/dist-test-taskkz56XN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127579900523-11321-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127579900523-11321-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127579900523-11321-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127579900523-11321-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.14.126:33525
--webserver_interface=127.11.14.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.126.keytab
--principal=kudu/127.11.14.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.14.84:45883
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.14.126:33525
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127579900523-11321-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127579900523-11321-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127579900523-11321-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:33:01.364117 11342 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:33:01.364465 11342 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:33:01.364583 11342 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:33:01.368871 11342 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:33:01.368978 11342 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:33:01.369021 11342 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:33:01.369082 11342 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:33:01.369124 11342 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:33:01.375000 11342 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.14.84:45883
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127579900523-11321-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127579900523-11321-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.14.126:33525
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.126.keytab
--principal=kudu/127.11.14.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.14.126:33525
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127579900523-11321-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.14.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11342
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127579900523-11321-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:33:01.376542 11342 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:33:01.377871 11342 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:33:01.391233 11350 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:33:01.392082 11347 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:33:01.392373 11342 server_base.cc:1034] running on GCE node
W20260807 18:33:01.392942 11348 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:33:01.393730 11342 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:33:01.395673 11342 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:33:01.396867 11342 hybrid_clock.cc:648] HybridClock initialized: now 1786127581396849 us; error 63 us; skew 500 ppm
Aug 07 18:33:01 dist-test-slave-6k22 krb5kdc[11328](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127581, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:33:01.399968 11342 init.cc:377] Logged in from keytab as kudu/127.11.14.126@KRBTEST.COM (short username kudu)
I20260807 18:33:01.401296 11342 webserver.cc:460] Webserver started at http://127.11.14.126:36813/ using document root <none> and password file <none>
I20260807 18:33:01.402177 11342 fs_manager.cc:359] Metadata directory not provided
I20260807 18:33:01.402246 11342 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:33:01.402571 11342 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:33:01.405304 11342 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127579900523-11321-0/minicluster-data/master-0/data/instance:
uuid: "fdfdd6906b8143b98c9d99c36a90520d"
format_stamp: "Formatted at 2026-08-07 18:33:01 on dist-test-slave-6k22"
server_key: "fa99f130f723a631d87c2eca51a442b0"
server_key_iv: "fbb26fbe210252a5d8bddeb2f227ca0c"
server_key_version: "encryptionkey@0"
I20260807 18:33:01.405889 11342 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127579900523-11321-0/minicluster-data/master-0/wal/instance:
uuid: "fdfdd6906b8143b98c9d99c36a90520d"
format_stamp: "Formatted at 2026-08-07 18:33:01 on dist-test-slave-6k22"
server_key: "fa99f130f723a631d87c2eca51a442b0"
server_key_iv: "fbb26fbe210252a5d8bddeb2f227ca0c"
server_key_version: "encryptionkey@0"
I20260807 18:33:01.411748 11342 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260807 18:33:01.415757 11357 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:33:01.417465 11342 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 18:33:01.417645 11342 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127579900523-11321-0/minicluster-data/master-0/data,/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127579900523-11321-0/minicluster-data/master-0/wal
uuid: "fdfdd6906b8143b98c9d99c36a90520d"
format_stamp: "Formatted at 2026-08-07 18:33:01 on dist-test-slave-6k22"
server_key: "fa99f130f723a631d87c2eca51a442b0"
server_key_iv: "fbb26fbe210252a5d8bddeb2f227ca0c"
server_key_version: "encryptionkey@0"
I20260807 18:33:01.417753 11342 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127579900523-11321-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127579900523-11321-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127579900523-11321-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:33:01.455732 11342 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:33:01.460135 11342 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:33:01.460404 11342 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:33:01.471369 11342 rpc_server.cc:307] RPC server started. Bound to: 127.11.14.126:33525
I20260807 18:33:01.471454 11413 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.14.126:33525 every 8 connection(s)
I20260807 18:33:01.472970 11342 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127579900523-11321-0/minicluster-data/master-0/data/info.pb
I20260807 18:33:01.475098 11321 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkz56XN/build/debug/bin/kudu as pid 11342
I20260807 18:33:01.475317 11321 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127579900523-11321-0/minicluster-data/master-0/wal/instance
I20260807 18:33:01.478042 11321 external_mini_cluster.cc:1428] Setting key d0b3db1add098c1bf25604e07b8e689a
I20260807 18:33:01.480299 11415 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:33:01.491186 11415 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fdfdd6906b8143b98c9d99c36a90520d: Bootstrap starting.
I20260807 18:33:01.495975 11415 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fdfdd6906b8143b98c9d99c36a90520d: Neither blocks nor log segments found. Creating new log.
I20260807 18:33:01.497416 11415 log.cc:824] T 00000000000000000000000000000000 P fdfdd6906b8143b98c9d99c36a90520d: Log is configured to *not* fsync() on all Append() calls
I20260807 18:33:01.500470 11415 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fdfdd6906b8143b98c9d99c36a90520d: No bootstrap required, opened a new log
Aug 07 18:33:01 dist-test-slave-6k22 krb5kdc[11328](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127581, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.14.126@KRBTEST.COM
I20260807 18:33:01.505362 11415 raft_consensus.cc:348] T 00000000000000000000000000000000 P fdfdd6906b8143b98c9d99c36a90520d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdfdd6906b8143b98c9d99c36a90520d" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 33525 } }
I20260807 18:33:01.505760 11415 raft_consensus.cc:374] T 00000000000000000000000000000000 P fdfdd6906b8143b98c9d99c36a90520d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:33:01.505868 11415 raft_consensus.cc:729] T 00000000000000000000000000000000 P fdfdd6906b8143b98c9d99c36a90520d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fdfdd6906b8143b98c9d99c36a90520d, State: Initialized, Role: FOLLOWER
I20260807 18:33:01.506837 11415 consensus_queue.cc:260] T 00000000000000000000000000000000 P fdfdd6906b8143b98c9d99c36a90520d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdfdd6906b8143b98c9d99c36a90520d" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 33525 } }
I20260807 18:33:01.507077 11415 raft_consensus.cc:388] T 00000000000000000000000000000000 P fdfdd6906b8143b98c9d99c36a90520d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:33:01.507359 11415 raft_consensus.cc:482] T 00000000000000000000000000000000 P fdfdd6906b8143b98c9d99c36a90520d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:33:01.507580 11415 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fdfdd6906b8143b98c9d99c36a90520d [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:33:01.508977 11415 raft_consensus.cc:504] T 00000000000000000000000000000000 P fdfdd6906b8143b98c9d99c36a90520d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdfdd6906b8143b98c9d99c36a90520d" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 33525 } }
I20260807 18:33:01.509670 11415 leader_election.cc:302] T 00000000000000000000000000000000 P fdfdd6906b8143b98c9d99c36a90520d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fdfdd6906b8143b98c9d99c36a90520d; no voters:
I20260807 18:33:01.510156 11415 leader_election.cc:288] T 00000000000000000000000000000000 P fdfdd6906b8143b98c9d99c36a90520d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:33:01.510324 11420 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fdfdd6906b8143b98c9d99c36a90520d [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:33:01.510552 11420 raft_consensus.cc:686] T 00000000000000000000000000000000 P fdfdd6906b8143b98c9d99c36a90520d [term 1 LEADER]: Becoming Leader. State: Replica: fdfdd6906b8143b98c9d99c36a90520d, State: Running, Role: LEADER
I20260807 18:33:01.511063 11420 consensus_queue.cc:237] T 00000000000000000000000000000000 P fdfdd6906b8143b98c9d99c36a90520d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdfdd6906b8143b98c9d99c36a90520d" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 33525 } }
I20260807 18:33:01.512006 11415 sys_catalog.cc:564] T 00000000000000000000000000000000 P fdfdd6906b8143b98c9d99c36a90520d [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:33:01.513115 11417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:01.485081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50714 (local address 127.11.14.126:33525)
0807 18:33:01.485671 (+ 590us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:01.485699 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:01.488704 (+ 3005us) server_negotiation.cc:407] Connection header received
0807 18:33:01.492767 (+ 4063us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:01.492802 (+ 35us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:01.493403 (+ 601us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:01.493879 (+ 476us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:33:01.496518 (+ 2639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:01.498271 (+ 1753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:01.499836 (+ 1565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:01.500641 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:01.500872 (+ 231us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:01.505050 (+ 4178us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:33:01.505087 (+ 37us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:33:01.505104 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:33:01.505137 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:33:01.507997 (+ 2860us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:01.508587 (+ 590us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:01.508594 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:01.508600 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:01.508765 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:01.509125 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:01.509130 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:01.509132 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:01.509915 (+ 783us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:33:01.510190 (+ 275us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:01.511149 (+ 959us) server_negotiation.cc:299] Negotiation successful
0807 18:33:01.511618 (+ 469us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":345,"thread_start_us":150,"threads_started":1}
I20260807 18:33:01.513691 11422 sys_catalog.cc:455] T 00000000000000000000000000000000 P fdfdd6906b8143b98c9d99c36a90520d [sys.catalog]: SysCatalogTable state changed. Reason: New leader fdfdd6906b8143b98c9d99c36a90520d. Latest consensus state: current_term: 1 leader_uuid: "fdfdd6906b8143b98c9d99c36a90520d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdfdd6906b8143b98c9d99c36a90520d" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 33525 } } }
I20260807 18:33:01.513837 11422 sys_catalog.cc:458] T 00000000000000000000000000000000 P fdfdd6906b8143b98c9d99c36a90520d [sys.catalog]: This master's current role is: LEADER
I20260807 18:33:01.514798 11421 sys_catalog.cc:455] T 00000000000000000000000000000000 P fdfdd6906b8143b98c9d99c36a90520d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fdfdd6906b8143b98c9d99c36a90520d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdfdd6906b8143b98c9d99c36a90520d" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 33525 } } }
I20260807 18:33:01.514948 11421 sys_catalog.cc:458] T 00000000000000000000000000000000 P fdfdd6906b8143b98c9d99c36a90520d [sys.catalog]: This master's current role is: LEADER
I20260807 18:33:01.515143 11431 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:33:01.520149 11431 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:33:01.528862 11431 catalog_manager.cc:1347] Generated new cluster ID: d59918dfc76d4baf8de3d86d6ca4ed9f
I20260807 18:33:01.528957 11431 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:33:01.546742 11431 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:33:01.548297 11431 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:33:01.555378 11431 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fdfdd6906b8143b98c9d99c36a90520d: Generated new TSK 0
I20260807 18:33:01.556478 11431 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.14.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.14.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.14.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.65.keytab.
Entry for principal kudu/127.11.14.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.65.keytab.
I20260807 18:33:01.598763 11321 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkz56XN/build/debug/bin/kudu
/tmp/dist-test-taskkz56XN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127579900523-11321-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127579900523-11321-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127579900523-11321-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127579900523-11321-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.14.65:0
--local_ip_for_outbound_sockets=127.11.14.65
--webserver_interface=127.11.14.65
--webserver_port=0
--tserver_master_addrs=127.11.14.126:33525
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.65.keytab
--principal=kudu/127.11.14.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.14.84:45883
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127579900523-11321-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127579900523-11321-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127579900523-11321-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:33:01.749699 11444 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:33:01.750001 11444 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:33:01.750079 11444 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:33:01.754205 11444 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:33:01.754305 11444 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:33:01.754401 11444 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.14.65
I20260807 18:33:01.759876 11444 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.14.84:45883
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127579900523-11321-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127579900523-11321-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.65.keytab
--principal=kudu/127.11.14.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.14.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127579900523-11321-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.14.65
--webserver_port=0
--tserver_master_addrs=127.11.14.126:33525
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11444
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.14.65
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127579900523-11321-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:33:01.761147 11444 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:33:01.762302 11444 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:33:01.773145 11450 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:33:01.773224 11449 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:33:01.773211 11452 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:33:01.773591 11444 server_base.cc:1034] running on GCE node
I20260807 18:33:01.774041 11444 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:33:01.774736 11444 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:33:01.775941 11444 hybrid_clock.cc:648] HybridClock initialized: now 1786127581775918 us; error 29 us; skew 500 ppm
Aug 07 18:33:01 dist-test-slave-6k22 krb5kdc[11328](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127581, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:33:01.779856 11444 init.cc:377] Logged in from keytab as kudu/127.11.14.65@KRBTEST.COM (short username kudu)
I20260807 18:33:01.780825 11444 webserver.cc:460] Webserver started at http://127.11.14.65:35433/ using document root <none> and password file <none>
I20260807 18:33:01.781447 11444 fs_manager.cc:359] Metadata directory not provided
I20260807 18:33:01.781544 11444 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:33:01.781805 11444 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:33:01.784276 11444 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127579900523-11321-0/minicluster-data/ts-0/data/instance:
uuid: "7be9e5ef2f2f485490971223849e94b7"
format_stamp: "Formatted at 2026-08-07 18:33:01 on dist-test-slave-6k22"
server_key: "86a355ae10abd45226d1113fd88257a5"
server_key_iv: "911d33fc338b68b723af2b39ecafcffa"
server_key_version: "encryptionkey@0"
I20260807 18:33:01.784931 11444 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127579900523-11321-0/minicluster-data/ts-0/wal/instance:
uuid: "7be9e5ef2f2f485490971223849e94b7"
format_stamp: "Formatted at 2026-08-07 18:33:01 on dist-test-slave-6k22"
server_key: "86a355ae10abd45226d1113fd88257a5"
server_key_iv: "911d33fc338b68b723af2b39ecafcffa"
server_key_version: "encryptionkey@0"
I20260807 18:33:01.789389 11444 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.003s
I20260807 18:33:01.792083 11460 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:33:01.793419 11444 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:33:01.793576 11444 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127579900523-11321-0/minicluster-data/ts-0/data,/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127579900523-11321-0/minicluster-data/ts-0/wal
uuid: "7be9e5ef2f2f485490971223849e94b7"
format_stamp: "Formatted at 2026-08-07 18:33:01 on dist-test-slave-6k22"
server_key: "86a355ae10abd45226d1113fd88257a5"
server_key_iv: "911d33fc338b68b723af2b39ecafcffa"
server_key_version: "encryptionkey@0"
I20260807 18:33:01.793720 11444 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127579900523-11321-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127579900523-11321-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127579900523-11321-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:33:01.814553 11444 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:33:01.818506 11444 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:33:01.818818 11444 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:33:01.819730 11444 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:33:01.821127 11444 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:33:01.821214 11444 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:33:01.821295 11444 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:33:01.821345 11444 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:33:01.839476 11444 rpc_server.cc:307] RPC server started. Bound to: 127.11.14.65:34525
I20260807 18:33:01.839605 11587 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.14.65:34525 every 8 connection(s)
I20260807 18:33:01.840685 11444 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127579900523-11321-0/minicluster-data/ts-0/data/info.pb
I20260807 18:33:01.847834 11321 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkz56XN/build/debug/bin/kudu as pid 11444
I20260807 18:33:01.847934 11321 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127579900523-11321-0/minicluster-data/ts-0/wal/instance
I20260807 18:33:01.848201 11321 external_mini_cluster.cc:1428] Setting key ac897f843a81fe780cfb3b15f2a87d8f
Aug 07 18:33:01 dist-test-slave-6k22 krb5kdc[11328](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127581, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.65@KRBTEST.COM for kudu/127.11.14.126@KRBTEST.COM
I20260807 18:33:01.857811 11417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:01.843121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.65:53053 (local address 127.11.14.126:33525)
0807 18:33:01.843270 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:01.843277 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:01.844828 (+ 1551us) server_negotiation.cc:407] Connection header received
0807 18:33:01.845941 (+ 1113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:01.845945 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:01.846014 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:01.846122 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:33:01.847608 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:01.848357 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:01.850178 (+ 1821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:01.850452 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:01.850574 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:01.853700 (+ 3126us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:33:01.853703 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:33:01.853706 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:33:01.853732 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:33:01.855223 (+ 1491us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:01.855965 (+ 742us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:01.855968 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:01.855969 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:01.856048 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:01.856490 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:01.856492 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:01.856493 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:01.856673 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:33:01.856748 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:01.857578 (+ 830us) server_negotiation.cc:299] Negotiation successful
0807 18:33:01.857686 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260807 18:33:01.859715 11590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:01.843502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:33525 (local address 127.11.14.65:53053)
0807 18:33:01.844580 (+ 1078us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:01.844695 (+ 115us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:01.845688 (+ 993us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:01.846323 (+ 635us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:01.846331 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:01.846793 (+ 462us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:33:01.847388 (+ 595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:01.847405 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:01.848523 (+ 1118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:01.848527 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:01.850024 (+ 1497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:01.850035 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:01.850619 (+ 584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:01.850624 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:01.850831 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:01.851480 (+ 649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:33:01.851499 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:33:01.853437 (+ 1938us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:33:01.855360 (+ 1923us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:33:01.855370 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:33:01.855391 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:33:01.855857 (+ 466us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:33:01.856157 (+ 300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:33:01.856163 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:33:01.856168 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:33:01.856384 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:33:01.856789 (+ 405us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:33:01.856811 (+ 22us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:33:01.857253 (+ 442us) client_negotiation.cc:769] Sending connection context
0807 18:33:01.857540 (+ 287us) client_negotiation.cc:240] Negotiation successful
0807 18:33:01.858109 (+ 569us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":825,"thread_start_us":221,"threads_started":1}
I20260807 18:33:01.861104 11588 heartbeater.cc:347] Connected to a master server at 127.11.14.126:33525
I20260807 18:33:01.861389 11588 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.11.14.66@KRBTEST.COM; defaulting to no policy
I20260807 18:33:01.861948 11588 heartbeater.cc:511] Master 127.11.14.126:33525 requested a full tablet report, sending...
I20260807 18:33:01.864015 11375 ts_manager.cc:194] Registered new tserver with Master: 7be9e5ef2f2f485490971223849e94b7 (127.11.14.65:34525)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.14.66@KRBTEST.COM" created.
I20260807 18:33:01.865622 11375 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.14.65@KRBTEST.COM'} at 127.11.14.65:53053
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.14.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.66.keytab.
Entry for principal kudu/127.11.14.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.66.keytab.
I20260807 18:33:01.882860 11321 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkz56XN/build/debug/bin/kudu
/tmp/dist-test-taskkz56XN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127579900523-11321-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127579900523-11321-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127579900523-11321-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127579900523-11321-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.14.66:0
--local_ip_for_outbound_sockets=127.11.14.66
--webserver_interface=127.11.14.66
--webserver_port=0
--tserver_master_addrs=127.11.14.126:33525
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.66.keytab
--principal=kudu/127.11.14.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.14.84:45883
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127579900523-11321-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127579900523-11321-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127579900523-11321-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:33:02.033979 11593 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:33:02.034274 11593 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:33:02.034358 11593 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:33:02.038481 11593 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:33:02.038568 11593 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:33:02.038655 11593 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.14.66
I20260807 18:33:02.044178 11593 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.14.84:45883
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127579900523-11321-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127579900523-11321-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.66.keytab
--principal=kudu/127.11.14.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.14.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127579900523-11321-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.14.66
--webserver_port=0
--tserver_master_addrs=127.11.14.126:33525
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11593
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.14.66
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127579900523-11321-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:33:02.045310 11593 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:33:02.046346 11593 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:33:02.058070 11602 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:33:02.058022 11598 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:33:02.058095 11593 server_base.cc:1034] running on GCE node
W20260807 18:33:02.058043 11599 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:33:02.058779 11593 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:33:02.059473 11593 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:33:02.060763 11593 hybrid_clock.cc:648] HybridClock initialized: now 1786127582060731 us; error 42 us; skew 500 ppm
Aug 07 18:33:02 dist-test-slave-6k22 krb5kdc[11328](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127582, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:33:02.063808 11593 init.cc:377] Logged in from keytab as kudu/127.11.14.66@KRBTEST.COM (short username kudu)
I20260807 18:33:02.064720 11593 webserver.cc:460] Webserver started at http://127.11.14.66:40565/ using document root <none> and password file <none>
I20260807 18:33:02.065323 11593 fs_manager.cc:359] Metadata directory not provided
I20260807 18:33:02.065418 11593 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:33:02.065676 11593 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:33:02.067873 11593 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127579900523-11321-0/minicluster-data/ts-1/data/instance:
uuid: "2ac05c9feffe445fab3fe13392426ea0"
format_stamp: "Formatted at 2026-08-07 18:33:02 on dist-test-slave-6k22"
server_key: "96aed299f78440d4692b46921a2e70fd"
server_key_iv: "d00a6a89b655ef1798b1e21f01ed5796"
server_key_version: "encryptionkey@0"
I20260807 18:33:02.068496 11593 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127579900523-11321-0/minicluster-data/ts-1/wal/instance:
uuid: "2ac05c9feffe445fab3fe13392426ea0"
format_stamp: "Formatted at 2026-08-07 18:33:02 on dist-test-slave-6k22"
server_key: "96aed299f78440d4692b46921a2e70fd"
server_key_iv: "d00a6a89b655ef1798b1e21f01ed5796"
server_key_version: "encryptionkey@0"
I20260807 18:33:02.072770 11593 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 18:33:02.075501 11610 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:33:02.076874 11593 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:33:02.077025 11593 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127579900523-11321-0/minicluster-data/ts-1/data,/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127579900523-11321-0/minicluster-data/ts-1/wal
uuid: "2ac05c9feffe445fab3fe13392426ea0"
format_stamp: "Formatted at 2026-08-07 18:33:02 on dist-test-slave-6k22"
server_key: "96aed299f78440d4692b46921a2e70fd"
server_key_iv: "d00a6a89b655ef1798b1e21f01ed5796"
server_key_version: "encryptionkey@0"
I20260807 18:33:02.077139 11593 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127579900523-11321-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127579900523-11321-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127579900523-11321-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:33:02.090224 11593 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:33:02.093817 11593 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:33:02.094036 11593 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:33:02.094797 11593 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:33:02.095721 11593 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:33:02.095786 11593 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:33:02.095865 11593 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:33:02.095903 11593 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:33:02.112643 11593 rpc_server.cc:307] RPC server started. Bound to: 127.11.14.66:33019
I20260807 18:33:02.112802 11733 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.14.66:33019 every 8 connection(s)
I20260807 18:33:02.113821 11593 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127579900523-11321-0/minicluster-data/ts-1/data/info.pb
I20260807 18:33:02.121049 11321 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkz56XN/build/debug/bin/kudu as pid 11593
I20260807 18:33:02.121162 11321 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127579900523-11321-0/minicluster-data/ts-1/wal/instance
I20260807 18:33:02.121425 11321 external_mini_cluster.cc:1428] Setting key bc84f8b3ddae6afe43016cb830045ad7
Aug 07 18:33:02 dist-test-slave-6k22 krb5kdc[11328](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127582, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.66@KRBTEST.COM for kudu/127.11.14.126@KRBTEST.COM
I20260807 18:33:02.130252 11417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:02.116028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.66:52489 (local address 127.11.14.126:33525)
0807 18:33:02.116207 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:02.116211 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:02.117471 (+ 1260us) server_negotiation.cc:407] Connection header received
0807 18:33:02.118673 (+ 1202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:02.118677 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:02.118743 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:02.118848 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:33:02.120312 (+ 1464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:02.120900 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:02.122323 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:02.122568 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:02.122729 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:02.125850 (+ 3121us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:33:02.125854 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:33:02.125856 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:33:02.125887 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:33:02.127711 (+ 1824us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:02.128348 (+ 637us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:02.128356 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:02.128360 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:02.128476 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:02.128888 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:02.128893 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:02.128895 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:02.129189 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:33:02.129325 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:02.129837 (+ 512us) server_negotiation.cc:299] Negotiation successful
0807 18:33:02.129992 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260807 18:33:02.130937 11736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:02.116463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:33525 (local address 127.11.14.66:52489)
0807 18:33:02.117306 (+ 843us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:02.117351 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:02.118385 (+ 1034us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:02.119033 (+ 648us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:02.119040 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:02.119447 (+ 407us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:33:02.120156 (+ 709us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:02.120173 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:02.121041 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:02.121051 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:02.122138 (+ 1087us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:02.122194 (+ 56us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:02.122758 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:02.122764 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:02.122957 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:02.123678 (+ 721us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:33:02.123696 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:33:02.125676 (+ 1980us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:33:02.127856 (+ 2180us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:33:02.127862 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:33:02.127878 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:33:02.128185 (+ 307us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:33:02.128606 (+ 421us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:33:02.128609 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:33:02.128610 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:33:02.128779 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:33:02.129308 (+ 529us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:33:02.129313 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:33:02.129566 (+ 253us) client_negotiation.cc:769] Sending connection context
0807 18:33:02.129728 (+ 162us) client_negotiation.cc:240] Negotiation successful
0807 18:33:02.129995 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":620,"thread_start_us":171,"threads_started":1}
I20260807 18:33:02.132309 11734 heartbeater.cc:347] Connected to a master server at 127.11.14.126:33525
I20260807 18:33:02.132613 11734 heartbeater.cc:464] Registering TS with master...
I20260807 18:33:02.133293 11734 heartbeater.cc:511] Master 127.11.14.126:33525 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.11.14.67@KRBTEST.COM; defaulting to no policy
I20260807 18:33:02.134582 11375 ts_manager.cc:194] Registered new tserver with Master: 2ac05c9feffe445fab3fe13392426ea0 (127.11.14.66:33019)
I20260807 18:33:02.135178 11375 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.14.66@KRBTEST.COM'} at 127.11.14.66:52489
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.14.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.14.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.67.keytab.
Entry for principal kudu/127.11.14.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.67.keytab.
I20260807 18:33:02.155095 11321 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkz56XN/build/debug/bin/kudu
/tmp/dist-test-taskkz56XN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127579900523-11321-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127579900523-11321-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127579900523-11321-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127579900523-11321-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.14.67:0
--local_ip_for_outbound_sockets=127.11.14.67
--webserver_interface=127.11.14.67
--webserver_port=0
--tserver_master_addrs=127.11.14.126:33525
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.67.keytab
--principal=kudu/127.11.14.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.14.84:45883
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127579900523-11321-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127579900523-11321-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127579900523-11321-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:33:02.301934 11739 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:33:02.302244 11739 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:33:02.302333 11739 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:33:02.306536 11739 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:33:02.306623 11739 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:33:02.306720 11739 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.14.67
I20260807 18:33:02.312120 11739 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.14.84:45883
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127579900523-11321-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127579900523-11321-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.67.keytab
--principal=kudu/127.11.14.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.14.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127579900523-11321-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.14.67
--webserver_port=0
--tserver_master_addrs=127.11.14.126:33525
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11739
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.14.67
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127579900523-11321-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:33:02.313265 11739 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:33:02.314301 11739 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:33:02.325197 11752 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:33:02.325119 11750 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:33:02.325120 11749 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:33:02.325532 11739 server_base.cc:1034] running on GCE node
I20260807 18:33:02.325956 11739 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:33:02.326642 11739 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:33:02.327800 11739 hybrid_clock.cc:648] HybridClock initialized: now 1786127582327786 us; error 26 us; skew 500 ppm
Aug 07 18:33:02 dist-test-slave-6k22 krb5kdc[11328](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127582, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:33:02.330531 11739 init.cc:377] Logged in from keytab as kudu/127.11.14.67@KRBTEST.COM (short username kudu)
I20260807 18:33:02.331403 11739 webserver.cc:460] Webserver started at http://127.11.14.67:42891/ using document root <none> and password file <none>
I20260807 18:33:02.332032 11739 fs_manager.cc:359] Metadata directory not provided
I20260807 18:33:02.332119 11739 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:33:02.332361 11739 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:33:02.334441 11739 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127579900523-11321-0/minicluster-data/ts-2/data/instance:
uuid: "86e4b99b85774ae185ceec6eff778ebf"
format_stamp: "Formatted at 2026-08-07 18:33:02 on dist-test-slave-6k22"
server_key: "4e8d1c0fc9c98e3168c10c78f20902dd"
server_key_iv: "7d763e854bf680658fc28663ee950532"
server_key_version: "encryptionkey@0"
I20260807 18:33:02.335023 11739 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127579900523-11321-0/minicluster-data/ts-2/wal/instance:
uuid: "86e4b99b85774ae185ceec6eff778ebf"
format_stamp: "Formatted at 2026-08-07 18:33:02 on dist-test-slave-6k22"
server_key: "4e8d1c0fc9c98e3168c10c78f20902dd"
server_key_iv: "7d763e854bf680658fc28663ee950532"
server_key_version: "encryptionkey@0"
I20260807 18:33:02.339041 11739 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 18:33:02.341778 11759 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:33:02.343079 11739 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 18:33:02.343216 11739 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127579900523-11321-0/minicluster-data/ts-2/data,/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127579900523-11321-0/minicluster-data/ts-2/wal
uuid: "86e4b99b85774ae185ceec6eff778ebf"
format_stamp: "Formatted at 2026-08-07 18:33:02 on dist-test-slave-6k22"
server_key: "4e8d1c0fc9c98e3168c10c78f20902dd"
server_key_iv: "7d763e854bf680658fc28663ee950532"
server_key_version: "encryptionkey@0"
I20260807 18:33:02.343330 11739 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127579900523-11321-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127579900523-11321-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127579900523-11321-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:33:02.355639 11739 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:33:02.358685 11739 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:33:02.358947 11739 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:33:02.359694 11739 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:33:02.360564 11739 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:33:02.360633 11739 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:33:02.360697 11739 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:33:02.360737 11739 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:33:02.377096 11880 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.14.67:43205 every 8 connection(s)
I20260807 18:33:02.377099 11739 rpc_server.cc:307] RPC server started. Bound to: 127.11.14.67:43205
I20260807 18:33:02.378168 11739 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127579900523-11321-0/minicluster-data/ts-2/data/info.pb
I20260807 18:33:02.382537 11321 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkz56XN/build/debug/bin/kudu as pid 11739
I20260807 18:33:02.382665 11321 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786127579900523-11321-0/minicluster-data/ts-2/wal/instance
I20260807 18:33:02.382963 11321 external_mini_cluster.cc:1428] Setting key 64a73625e3e3a41b42eb2652d82328f7
Aug 07 18:33:02 dist-test-slave-6k22 krb5kdc[11328](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127582, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.67@KRBTEST.COM for kudu/127.11.14.126@KRBTEST.COM
I20260807 18:33:02.392683 11417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:02.380115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.67:32969 (local address 127.11.14.126:33525)
0807 18:33:02.380290 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:02.380312 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:02.381523 (+ 1211us) server_negotiation.cc:407] Connection header received
0807 18:33:02.382589 (+ 1066us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:02.382595 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:02.382659 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:02.382770 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:33:02.384208 (+ 1438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:02.384841 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:02.386323 (+ 1482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:02.386545 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:02.386631 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:02.389242 (+ 2611us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:33:02.389246 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:33:02.389248 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:33:02.389272 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:33:02.390711 (+ 1439us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:02.391238 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:02.391240 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:02.391242 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:02.391316 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:02.391674 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:02.391676 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:02.391677 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:02.391861 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:33:02.391937 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:02.392459 (+ 522us) server_negotiation.cc:299] Negotiation successful
0807 18:33:02.392559 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260807 18:33:02.393575 11883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:02.380578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:33525 (local address 127.11.14.67:32969)
0807 18:33:02.381286 (+ 708us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:02.381363 (+ 77us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:02.382371 (+ 1008us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:02.382927 (+ 556us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:02.382933 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:02.383326 (+ 393us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:33:02.384054 (+ 728us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:02.384080 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:02.384986 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:02.384989 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:02.386209 (+ 1220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:02.386220 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:02.386682 (+ 462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:02.386685 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:02.386834 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:02.387478 (+ 644us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:33:02.387495 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:33:02.389072 (+ 1577us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:33:02.390829 (+ 1757us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:33:02.390835 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:33:02.390847 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:33:02.391142 (+ 295us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:33:02.391389 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:33:02.391391 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:33:02.391392 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:33:02.391528 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:33:02.391957 (+ 429us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:33:02.391962 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:33:02.392243 (+ 281us) client_negotiation.cc:769] Sending connection context
0807 18:33:02.392399 (+ 156us) client_negotiation.cc:240] Negotiation successful
0807 18:33:02.392672 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":468,"thread_start_us":103,"threads_started":1}
I20260807 18:33:02.394773 11881 heartbeater.cc:347] Connected to a master server at 127.11.14.126:33525
I20260807 18:33:02.395005 11881 heartbeater.cc:464] Registering TS with master...
I20260807 18:33:02.395488 11881 heartbeater.cc:511] Master 127.11.14.126:33525 requested a full tablet report, sending...
I20260807 18:33:02.396670 11375 ts_manager.cc:194] Registered new tserver with Master: 86e4b99b85774ae185ceec6eff778ebf (127.11.14.67:43205)
I20260807 18:33:02.397352 11375 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.14.67@KRBTEST.COM'} at 127.11.14.67:32969
I20260807 18:33:02.397828 11321 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 18:33:02 dist-test-slave-6k22 krb5kdc[11328](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127582, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 07 18:33:02 dist-test-slave-6k22 krb5kdc[11328](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127582, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.11.14.65@KRBTEST.COM
I20260807 18:33:02.426796 11890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:02.417050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42834 (local address 127.11.14.65:34525)
0807 18:33:02.417367 (+ 317us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:02.417373 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:02.417399 (+ 26us) server_negotiation.cc:407] Connection header received
0807 18:33:02.417503 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:02.417508 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:02.417756 (+ 248us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:02.417883 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:33:02.418596 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:02.419397 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:02.420271 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:02.420681 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:02.420770 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:02.423228 (+ 2458us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:33:02.423237 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:33:02.423252 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:33:02.423275 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:33:02.424956 (+ 1681us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:02.425359 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:02.425364 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:02.425365 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:02.425446 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:02.425752 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:02.425754 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:02.425755 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:02.425990 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:33:02.426155 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:02.426395 (+ 240us) server_negotiation.cc:299] Negotiation successful
0807 18:33:02.426593 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":216,"thread_start_us":91,"threads_started":1}
W20260807 18:33:02.427986 11492 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42834
Aug 07 18:33:02 dist-test-slave-6k22 krb5kdc[11328](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127582, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 07 18:33:02 dist-test-slave-6k22 krb5kdc[11328](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127582, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.14.65@KRBTEST.COM
I20260807 18:33:02.456017 11890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:02.447543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42848 (local address 127.11.14.65:34525)
0807 18:33:02.447770 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:02.447775 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:02.447892 (+ 117us) server_negotiation.cc:407] Connection header received
0807 18:33:02.448066 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:02.448071 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:02.448131 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:02.448226 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:33:02.448915 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:02.449426 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:02.450254 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:02.450492 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:02.450598 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:02.452972 (+ 2374us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:33:02.452976 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:33:02.452984 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:33:02.453007 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:33:02.454330 (+ 1323us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:02.454759 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:02.454763 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:02.454765 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:02.454846 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:02.455117 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:02.455123 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:02.455128 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:02.455297 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:33:02.455438 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:02.455723 (+ 285us) server_negotiation.cc:299] Negotiation successful
0807 18:33:02.455848 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124}
I20260807 18:33:02.457157 11321 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkz56XN/build/debug/bin/kudu with pid 11444
I20260807 18:33:02.469925 11321 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkz56XN/build/debug/bin/kudu with pid 11593
I20260807 18:33:02.477653 11321 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkz56XN/build/debug/bin/kudu with pid 11739
I20260807 18:33:02.489017 11321 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkz56XN/build/debug/bin/kudu with pid 11342
2026-08-07T18:33:02Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2555 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127579900523-11321-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:33:02 dist-test-slave-6k22 krb5kdc[11897](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:33:02 dist-test-slave-6k22 krb5kdc[11897](info): set up 2 sockets
Aug 07 18:33:02 dist-test-slave-6k22 krb5kdc[11897](info): commencing operation
krb5kdc: starting...
W20260807 18:33:03.655681 11321 mini_kdc.cc:121] Time spent starting KDC: real 1.142s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127579900523-11321-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127579900523-11321-0/krb5kdc/test-user.keytab.
Aug 07 18:33:03 dist-test-slave-6k22 krb5kdc[11897](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127583, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:33:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:33:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.14.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.14.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.14.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.126.keytab.
Entry for principal kudu/127.11.14.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.126.keytab.
I20260807 18:33:03.840262 11321 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkz56XN/build/debug/bin/kudu
/tmp/dist-test-taskkz56XN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127579900523-11321-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127579900523-11321-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127579900523-11321-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127579900523-11321-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.14.126:39415
--webserver_interface=127.11.14.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.126.keytab
--principal=kudu/127.11.14.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.14.84:43881
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.14.126:39415
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127579900523-11321-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127579900523-11321-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127579900523-11321-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:33:04.020864 11910 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:33:04.021349 11910 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:33:04.021445 11910 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:33:04.026274 11910 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:33:04.026345 11910 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:33:04.026365 11910 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:33:04.026382 11910 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:33:04.026398 11910 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:33:04.033702 11910 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.14.84:43881
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127579900523-11321-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127579900523-11321-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.14.126:39415
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.126.keytab
--principal=kudu/127.11.14.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.14.126:39415
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127579900523-11321-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.14.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11910
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127579900523-11321-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:33:04.035581 11910 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:33:04.038183 11910 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:33:04.050006 11915 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:33:04.050009 11918 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:33:04.050006 11916 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:33:04.050388 11910 server_base.cc:1034] running on GCE node
I20260807 18:33:04.051544 11910 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:33:04.052872 11910 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:33:04.054085 11910 hybrid_clock.cc:648] HybridClock initialized: now 1786127584054059 us; error 38 us; skew 500 ppm
Aug 07 18:33:04 dist-test-slave-6k22 krb5kdc[11897](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127584, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:33:04.057679 11910 init.cc:377] Logged in from keytab as kudu/127.11.14.126@KRBTEST.COM (short username kudu)
I20260807 18:33:04.058797 11910 webserver.cc:460] Webserver started at http://127.11.14.126:40497/ using document root <none> and password file <none>
I20260807 18:33:04.059800 11910 fs_manager.cc:359] Metadata directory not provided
I20260807 18:33:04.059896 11910 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:33:04.060158 11910 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:33:04.062635 11910 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127579900523-11321-0/minicluster-data/master-0/data/instance:
uuid: "e0c7c908d26f4c74b65ec615ea56763f"
format_stamp: "Formatted at 2026-08-07 18:33:04 on dist-test-slave-6k22"
server_key: "5d0cb4083c680b4b0b7242960cd3009e"
server_key_iv: "a154a13605985573deec00e7fd05d7a4"
server_key_version: "encryptionkey@0"
I20260807 18:33:04.063860 11910 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127579900523-11321-0/minicluster-data/master-0/wal/instance:
uuid: "e0c7c908d26f4c74b65ec615ea56763f"
format_stamp: "Formatted at 2026-08-07 18:33:04 on dist-test-slave-6k22"
server_key: "5d0cb4083c680b4b0b7242960cd3009e"
server_key_iv: "a154a13605985573deec00e7fd05d7a4"
server_key_version: "encryptionkey@0"
I20260807 18:33:04.068619 11910 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260807 18:33:04.071244 11926 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:33:04.072903 11910 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260807 18:33:04.073208 11910 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127579900523-11321-0/minicluster-data/master-0/data,/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127579900523-11321-0/minicluster-data/master-0/wal
uuid: "e0c7c908d26f4c74b65ec615ea56763f"
format_stamp: "Formatted at 2026-08-07 18:33:04 on dist-test-slave-6k22"
server_key: "5d0cb4083c680b4b0b7242960cd3009e"
server_key_iv: "a154a13605985573deec00e7fd05d7a4"
server_key_version: "encryptionkey@0"
I20260807 18:33:04.073372 11910 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127579900523-11321-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127579900523-11321-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127579900523-11321-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:33:04.098712 11910 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:33:04.103009 11910 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:33:04.103241 11910 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:33:04.117349 11910 rpc_server.cc:307] RPC server started. Bound to: 127.11.14.126:39415
I20260807 18:33:04.117408 11982 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.14.126:39415 every 8 connection(s)
I20260807 18:33:04.118660 11910 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127579900523-11321-0/minicluster-data/master-0/data/info.pb
I20260807 18:33:04.123605 11321 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkz56XN/build/debug/bin/kudu as pid 11910
I20260807 18:33:04.123744 11321 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127579900523-11321-0/minicluster-data/master-0/wal/instance
I20260807 18:33:04.124013 11983 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:33:04.124224 11321 external_mini_cluster.cc:1428] Setting key 77269e2216422161215868bc26f92ab4
I20260807 18:33:04.141640 11983 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e0c7c908d26f4c74b65ec615ea56763f: Bootstrap starting.
Aug 07 18:33:04 dist-test-slave-6k22 krb5kdc[11897](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127583, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.14.126@KRBTEST.COM
I20260807 18:33:04.146350 11983 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e0c7c908d26f4c74b65ec615ea56763f: Neither blocks nor log segments found. Creating new log.
I20260807 18:33:04.147990 11983 log.cc:824] T 00000000000000000000000000000000 P e0c7c908d26f4c74b65ec615ea56763f: Log is configured to *not* fsync() on all Append() calls
I20260807 18:33:04.150694 11983 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e0c7c908d26f4c74b65ec615ea56763f: No bootstrap required, opened a new log
I20260807 18:33:04.152431 11986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:04.126930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60078 (local address 127.11.14.126:39415)
0807 18:33:04.127534 (+ 604us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:04.127585 (+ 51us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:04.127635 (+ 50us) server_negotiation.cc:407] Connection header received
0807 18:33:04.128903 (+ 1268us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:04.128957 (+ 54us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:04.130077 (+ 1120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:04.130583 (+ 506us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:33:04.131742 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:04.134098 (+ 2356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:04.136270 (+ 2172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:04.140345 (+ 4075us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:04.141257 (+ 912us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:04.145341 (+ 4084us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:33:04.145367 (+ 26us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:33:04.145441 (+ 74us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:33:04.145575 (+ 134us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:33:04.148687 (+ 3112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:04.149260 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:04.149271 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:04.149277 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:04.149400 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:04.149763 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:04.149766 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:04.149767 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:04.150351 (+ 584us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:33:04.150661 (+ 310us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:04.151170 (+ 509us) server_negotiation.cc:299] Negotiation successful
0807 18:33:04.151510 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":322,"thread_start_us":132,"threads_started":1}
I20260807 18:33:04.157544 11983 raft_consensus.cc:348] T 00000000000000000000000000000000 P e0c7c908d26f4c74b65ec615ea56763f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0c7c908d26f4c74b65ec615ea56763f" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 39415 } }
I20260807 18:33:04.157910 11983 raft_consensus.cc:374] T 00000000000000000000000000000000 P e0c7c908d26f4c74b65ec615ea56763f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:33:04.157989 11983 raft_consensus.cc:729] T 00000000000000000000000000000000 P e0c7c908d26f4c74b65ec615ea56763f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e0c7c908d26f4c74b65ec615ea56763f, State: Initialized, Role: FOLLOWER
I20260807 18:33:04.158453 11983 consensus_queue.cc:260] T 00000000000000000000000000000000 P e0c7c908d26f4c74b65ec615ea56763f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0c7c908d26f4c74b65ec615ea56763f" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 39415 } }
I20260807 18:33:04.158600 11983 raft_consensus.cc:388] T 00000000000000000000000000000000 P e0c7c908d26f4c74b65ec615ea56763f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:33:04.158665 11983 raft_consensus.cc:482] T 00000000000000000000000000000000 P e0c7c908d26f4c74b65ec615ea56763f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:33:04.158761 11983 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e0c7c908d26f4c74b65ec615ea56763f [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:33:04.159950 11983 raft_consensus.cc:504] T 00000000000000000000000000000000 P e0c7c908d26f4c74b65ec615ea56763f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0c7c908d26f4c74b65ec615ea56763f" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 39415 } }
I20260807 18:33:04.160317 11983 leader_election.cc:302] T 00000000000000000000000000000000 P e0c7c908d26f4c74b65ec615ea56763f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e0c7c908d26f4c74b65ec615ea56763f; no voters:
I20260807 18:33:04.160650 11983 leader_election.cc:288] T 00000000000000000000000000000000 P e0c7c908d26f4c74b65ec615ea56763f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:33:04.160794 11988 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e0c7c908d26f4c74b65ec615ea56763f [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:33:04.161163 11988 raft_consensus.cc:686] T 00000000000000000000000000000000 P e0c7c908d26f4c74b65ec615ea56763f [term 1 LEADER]: Becoming Leader. State: Replica: e0c7c908d26f4c74b65ec615ea56763f, State: Running, Role: LEADER
I20260807 18:33:04.162153 11988 consensus_queue.cc:237] T 00000000000000000000000000000000 P e0c7c908d26f4c74b65ec615ea56763f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0c7c908d26f4c74b65ec615ea56763f" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 39415 } }
I20260807 18:33:04.164047 11983 sys_catalog.cc:564] T 00000000000000000000000000000000 P e0c7c908d26f4c74b65ec615ea56763f [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:33:04.165226 11990 sys_catalog.cc:455] T 00000000000000000000000000000000 P e0c7c908d26f4c74b65ec615ea56763f [sys.catalog]: SysCatalogTable state changed. Reason: New leader e0c7c908d26f4c74b65ec615ea56763f. Latest consensus state: current_term: 1 leader_uuid: "e0c7c908d26f4c74b65ec615ea56763f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0c7c908d26f4c74b65ec615ea56763f" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 39415 } } }
I20260807 18:33:04.165438 11990 sys_catalog.cc:458] T 00000000000000000000000000000000 P e0c7c908d26f4c74b65ec615ea56763f [sys.catalog]: This master's current role is: LEADER
I20260807 18:33:04.165791 11989 sys_catalog.cc:455] T 00000000000000000000000000000000 P e0c7c908d26f4c74b65ec615ea56763f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e0c7c908d26f4c74b65ec615ea56763f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0c7c908d26f4c74b65ec615ea56763f" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 39415 } } }
I20260807 18:33:04.165946 11989 sys_catalog.cc:458] T 00000000000000000000000000000000 P e0c7c908d26f4c74b65ec615ea56763f [sys.catalog]: This master's current role is: LEADER
I20260807 18:33:04.165860 11997 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:33:04.169075 11997 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:33:04.181499 11997 catalog_manager.cc:1347] Generated new cluster ID: 129ed550f086414fb082946e6e05b643
I20260807 18:33:04.181998 11997 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:33:04.234113 11997 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:33:04.235500 11997 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:33:04.244791 11997 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e0c7c908d26f4c74b65ec615ea56763f: Generated new TSK 0
I20260807 18:33:04.245815 11997 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.14.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.14.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.14.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.65.keytab.
Entry for principal kudu/127.11.14.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.65.keytab.
I20260807 18:33:04.305163 11321 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkz56XN/build/debug/bin/kudu
/tmp/dist-test-taskkz56XN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127579900523-11321-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127579900523-11321-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127579900523-11321-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127579900523-11321-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.14.65:0
--local_ip_for_outbound_sockets=127.11.14.65
--webserver_interface=127.11.14.65
--webserver_port=0
--tserver_master_addrs=127.11.14.126:39415
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.65.keytab
--principal=kudu/127.11.14.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.14.84:43881
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127579900523-11321-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127579900523-11321-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127579900523-11321-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:33:04.584882 12010 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:33:04.585448 12010 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:33:04.585548 12010 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:33:04.590853 12010 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:33:04.590950 12010 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:33:04.591059 12010 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.14.65
I20260807 18:33:04.600968 12010 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.14.84:43881
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127579900523-11321-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127579900523-11321-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.65.keytab
--principal=kudu/127.11.14.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.14.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127579900523-11321-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.14.65
--webserver_port=0
--tserver_master_addrs=127.11.14.126:39415
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12010
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.14.65
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127579900523-11321-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:33:04.602782 12010 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:33:04.604156 12010 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:33:04.617022 12015 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:33:04.617041 12016 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:33:04.617025 12018 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:33:04.617450 12010 server_base.cc:1034] running on GCE node
I20260807 18:33:04.618070 12010 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:33:04.619102 12010 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:33:04.620289 12010 hybrid_clock.cc:648] HybridClock initialized: now 1786127584620252 us; error 58 us; skew 500 ppm
Aug 07 18:33:04 dist-test-slave-6k22 krb5kdc[11897](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127584, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:33:04.626255 12010 init.cc:377] Logged in from keytab as kudu/127.11.14.65@KRBTEST.COM (short username kudu)
I20260807 18:33:04.628325 12010 webserver.cc:460] Webserver started at http://127.11.14.65:40675/ using document root <none> and password file <none>
I20260807 18:33:04.629074 12010 fs_manager.cc:359] Metadata directory not provided
I20260807 18:33:04.629191 12010 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:33:04.629908 12010 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:33:04.632985 12010 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127579900523-11321-0/minicluster-data/ts-0/data/instance:
uuid: "1c688728860e473ba09b74079a76bff0"
format_stamp: "Formatted at 2026-08-07 18:33:04 on dist-test-slave-6k22"
server_key: "54c559e4e3d5bb4337e7d5a9075e2c94"
server_key_iv: "b6899a103ac3968d959de8ee89437fd1"
server_key_version: "encryptionkey@0"
I20260807 18:33:04.633729 12010 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127579900523-11321-0/minicluster-data/ts-0/wal/instance:
uuid: "1c688728860e473ba09b74079a76bff0"
format_stamp: "Formatted at 2026-08-07 18:33:04 on dist-test-slave-6k22"
server_key: "54c559e4e3d5bb4337e7d5a9075e2c94"
server_key_iv: "b6899a103ac3968d959de8ee89437fd1"
server_key_version: "encryptionkey@0"
I20260807 18:33:04.640687 12010 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.004s
I20260807 18:33:04.643740 12026 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:33:04.645424 12010 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 18:33:04.645569 12010 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127579900523-11321-0/minicluster-data/ts-0/data,/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127579900523-11321-0/minicluster-data/ts-0/wal
uuid: "1c688728860e473ba09b74079a76bff0"
format_stamp: "Formatted at 2026-08-07 18:33:04 on dist-test-slave-6k22"
server_key: "54c559e4e3d5bb4337e7d5a9075e2c94"
server_key_iv: "b6899a103ac3968d959de8ee89437fd1"
server_key_version: "encryptionkey@0"
I20260807 18:33:04.645706 12010 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127579900523-11321-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127579900523-11321-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127579900523-11321-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:33:04.674420 12010 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:33:04.679756 12010 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:33:04.680120 12010 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:33:04.681427 12010 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:33:04.684011 12010 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:33:04.684065 12010 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:33:04.684131 12010 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:33:04.684163 12010 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:33:04.702491 12010 rpc_server.cc:307] RPC server started. Bound to: 127.11.14.65:32989
I20260807 18:33:04.702584 12154 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.14.65:32989 every 8 connection(s)
I20260807 18:33:04.703835 12010 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127579900523-11321-0/minicluster-data/ts-0/data/info.pb
I20260807 18:33:04.711524 11321 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkz56XN/build/debug/bin/kudu as pid 12010
I20260807 18:33:04.711771 11321 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127579900523-11321-0/minicluster-data/ts-0/wal/instance
I20260807 18:33:04.712468 11321 external_mini_cluster.cc:1428] Setting key 7eef73cec9ff91691dcdff832d7406be
Aug 07 18:33:04 dist-test-slave-6k22 krb5kdc[11897](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127584, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.65@KRBTEST.COM for kudu/127.11.14.126@KRBTEST.COM
I20260807 18:33:04.730548 12157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:04.706930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.65:44099 (local address 127.11.14.126:39415)
0807 18:33:04.707287 (+ 357us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:04.707297 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:04.708720 (+ 1423us) server_negotiation.cc:407] Connection header received
0807 18:33:04.712400 (+ 3680us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:04.712416 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:04.712507 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:04.712661 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:33:04.714568 (+ 1907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:04.715233 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:04.717045 (+ 1812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:04.717342 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:04.717456 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:04.722838 (+ 5382us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:33:04.722879 (+ 41us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:33:04.722924 (+ 45us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:33:04.723014 (+ 90us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:33:04.725924 (+ 2910us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:04.726976 (+ 1052us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:04.726986 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:04.726988 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:04.727154 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:04.728297 (+ 1143us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:04.728314 (+ 17us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:04.728315 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:04.728700 (+ 385us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:33:04.728954 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:04.730204 (+ 1250us) server_negotiation.cc:299] Negotiation successful
0807 18:33:04.730351 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":265,"thread_start_us":172,"threads_started":1}
I20260807 18:33:04.732074 12159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:04.707474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:39415 (local address 127.11.14.65:44099)
0807 18:33:04.708385 (+ 911us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:04.708525 (+ 140us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:04.711773 (+ 3248us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:04.712916 (+ 1143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:04.712925 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:04.713552 (+ 627us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:33:04.714369 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:04.714400 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:04.715653 (+ 1253us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:04.715658 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:04.716914 (+ 1256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:04.716925 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:04.717556 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:04.717577 (+ 21us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:04.717782 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:04.718732 (+ 950us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:33:04.718750 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:33:04.722118 (+ 3368us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:33:04.726068 (+ 3950us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:33:04.726077 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:33:04.726123 (+ 46us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:33:04.726676 (+ 553us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:33:04.727413 (+ 737us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:33:04.727417 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:33:04.727427 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:33:04.727939 (+ 512us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:33:04.728924 (+ 985us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:33:04.728964 (+ 40us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:33:04.729787 (+ 823us) client_negotiation.cc:769] Sending connection context
0807 18:33:04.730149 (+ 362us) client_negotiation.cc:240] Negotiation successful
0807 18:33:04.730537 (+ 388us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":597,"thread_start_us":312,"threads_started":1}
WARNING: no policy specified for kudu/127.11.14.66@KRBTEST.COM; defaulting to no policy
I20260807 18:33:04.734222 12155 heartbeater.cc:347] Connected to a master server at 127.11.14.126:39415
I20260807 18:33:04.734596 12155 heartbeater.cc:464] Registering TS with master...
I20260807 18:33:04.735365 12155 heartbeater.cc:511] Master 127.11.14.126:39415 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.14.66@KRBTEST.COM" created.
I20260807 18:33:04.737340 11945 ts_manager.cc:194] Registered new tserver with Master: 1c688728860e473ba09b74079a76bff0 (127.11.14.65:32989)
I20260807 18:33:04.739058 11945 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.14.65@KRBTEST.COM'} at 127.11.14.65:44099
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.14.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.66.keytab.
Entry for principal kudu/127.11.14.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.66.keytab.
I20260807 18:33:04.764592 11321 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkz56XN/build/debug/bin/kudu
/tmp/dist-test-taskkz56XN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127579900523-11321-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127579900523-11321-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127579900523-11321-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127579900523-11321-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.14.66:0
--local_ip_for_outbound_sockets=127.11.14.66
--webserver_interface=127.11.14.66
--webserver_port=0
--tserver_master_addrs=127.11.14.126:39415
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.66.keytab
--principal=kudu/127.11.14.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.14.84:43881
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127579900523-11321-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127579900523-11321-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127579900523-11321-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:33:04.973775 12162 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:33:04.974114 12162 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:33:04.974165 12162 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:33:04.980276 12162 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:33:04.980471 12162 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:33:04.980609 12162 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.14.66
I20260807 18:33:04.992796 12162 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.14.84:43881
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127579900523-11321-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127579900523-11321-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.66.keytab
--principal=kudu/127.11.14.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.14.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127579900523-11321-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.14.66
--webserver_port=0
--tserver_master_addrs=127.11.14.126:39415
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12162
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.14.66
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127579900523-11321-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:33:04.994668 12162 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:33:04.996507 12162 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:33:05.012866 12168 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:33:05.012905 12171 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:33:05.013027 12167 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:33:05.014695 12162 server_base.cc:1034] running on GCE node
I20260807 18:33:05.015136 12162 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:33:05.015945 12162 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:33:05.019327 12162 hybrid_clock.cc:648] HybridClock initialized: now 1786127585019266 us; error 76 us; skew 500 ppm
Aug 07 18:33:05 dist-test-slave-6k22 krb5kdc[11897](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127585, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:33:05.031620 12162 init.cc:377] Logged in from keytab as kudu/127.11.14.66@KRBTEST.COM (short username kudu)
I20260807 18:33:05.032811 12162 webserver.cc:460] Webserver started at http://127.11.14.66:36377/ using document root <none> and password file <none>
I20260807 18:33:05.033569 12162 fs_manager.cc:359] Metadata directory not provided
I20260807 18:33:05.033676 12162 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:33:05.033981 12162 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:33:05.038338 12162 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127579900523-11321-0/minicluster-data/ts-1/data/instance:
uuid: "0b40acc5583a4f11b6f01ecb91adda7f"
format_stamp: "Formatted at 2026-08-07 18:33:05 on dist-test-slave-6k22"
server_key: "e7828f7af37bca54acb444b94c7a040b"
server_key_iv: "59b587ced0a17a0f4cb5653a8571844b"
server_key_version: "encryptionkey@0"
I20260807 18:33:05.039258 12162 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127579900523-11321-0/minicluster-data/ts-1/wal/instance:
uuid: "0b40acc5583a4f11b6f01ecb91adda7f"
format_stamp: "Formatted at 2026-08-07 18:33:05 on dist-test-slave-6k22"
server_key: "e7828f7af37bca54acb444b94c7a040b"
server_key_iv: "59b587ced0a17a0f4cb5653a8571844b"
server_key_version: "encryptionkey@0"
I20260807 18:33:05.046789 12162 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.005s sys 0.000s
I20260807 18:33:05.051697 12179 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:33:05.053354 12162 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 18:33:05.053505 12162 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127579900523-11321-0/minicluster-data/ts-1/data,/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127579900523-11321-0/minicluster-data/ts-1/wal
uuid: "0b40acc5583a4f11b6f01ecb91adda7f"
format_stamp: "Formatted at 2026-08-07 18:33:05 on dist-test-slave-6k22"
server_key: "e7828f7af37bca54acb444b94c7a040b"
server_key_iv: "59b587ced0a17a0f4cb5653a8571844b"
server_key_version: "encryptionkey@0"
I20260807 18:33:05.053711 12162 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127579900523-11321-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127579900523-11321-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127579900523-11321-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:33:05.069590 12162 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:33:05.073917 12162 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:33:05.074235 12162 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:33:05.075443 12162 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:33:05.076717 12162 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:33:05.076772 12162 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:33:05.076818 12162 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:33:05.076836 12162 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:33:05.098475 12162 rpc_server.cc:307] RPC server started. Bound to: 127.11.14.66:43123
I20260807 18:33:05.098522 12301 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.14.66:43123 every 8 connection(s)
I20260807 18:33:05.100044 12162 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127579900523-11321-0/minicluster-data/ts-1/data/info.pb
I20260807 18:33:05.109516 11321 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkz56XN/build/debug/bin/kudu as pid 12162
I20260807 18:33:05.109663 11321 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127579900523-11321-0/minicluster-data/ts-1/wal/instance
I20260807 18:33:05.110543 11321 external_mini_cluster.cc:1428] Setting key cda8a550d951e07e869e6e9366502e21
Aug 07 18:33:05 dist-test-slave-6k22 krb5kdc[11897](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127585, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.66@KRBTEST.COM for kudu/127.11.14.126@KRBTEST.COM
I20260807 18:33:05.130412 12157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:05.102720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.66:57711 (local address 127.11.14.126:39415)
0807 18:33:05.103130 (+ 410us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:05.103137 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:05.104516 (+ 1379us) server_negotiation.cc:407] Connection header received
0807 18:33:05.107808 (+ 3292us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:05.107812 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:05.107953 (+ 141us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:05.108127 (+ 174us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:33:05.111218 (+ 3091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:05.112095 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:05.114902 (+ 2807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:05.115830 (+ 928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:05.115991 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:05.122208 (+ 6217us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:33:05.122216 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:33:05.122223 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:33:05.122265 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:33:05.124998 (+ 2733us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:05.127133 (+ 2135us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:05.127140 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:05.127145 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:05.127269 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:05.127986 (+ 717us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:05.127991 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:05.127995 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:05.128452 (+ 457us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:33:05.128558 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:05.129936 (+ 1378us) server_negotiation.cc:299] Negotiation successful
0807 18:33:05.130150 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":210}
WARNING: no policy specified for kudu/127.11.14.67@KRBTEST.COM; defaulting to no policy
I20260807 18:33:05.132709 12304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:05.103629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:39415 (local address 127.11.14.66:57711)
0807 18:33:05.104340 (+ 711us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:05.104388 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:05.107220 (+ 2832us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:05.108484 (+ 1264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:05.108494 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:05.109506 (+ 1012us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:33:05.110846 (+ 1340us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:05.110871 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:05.112825 (+ 1954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:05.112881 (+ 56us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:05.114700 (+ 1819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:05.114725 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:05.116007 (+ 1282us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:05.116010 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:05.116221 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:05.116990 (+ 769us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:33:05.117011 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:33:05.121890 (+ 4879us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:33:05.125573 (+ 3683us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:33:05.125640 (+ 67us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:33:05.125704 (+ 64us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:33:05.126551 (+ 847us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:33:05.127432 (+ 881us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:33:05.127439 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:33:05.127443 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:33:05.127703 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:33:05.128794 (+ 1091us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:33:05.128806 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:33:05.129421 (+ 615us) client_negotiation.cc:769] Sending connection context
0807 18:33:05.130678 (+ 1257us) client_negotiation.cc:240] Negotiation successful
0807 18:33:05.131188 (+ 510us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":443,"thread_start_us":213,"threads_started":1}
I20260807 18:33:05.134639 12302 heartbeater.cc:347] Connected to a master server at 127.11.14.126:39415
I20260807 18:33:05.134995 12302 heartbeater.cc:464] Registering TS with master...
I20260807 18:33:05.135600 12302 heartbeater.cc:511] Master 127.11.14.126:39415 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.14.67@KRBTEST.COM" created.
I20260807 18:33:05.137168 11945 ts_manager.cc:194] Registered new tserver with Master: 0b40acc5583a4f11b6f01ecb91adda7f (127.11.14.66:43123)
I20260807 18:33:05.138255 11945 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.14.66@KRBTEST.COM'} at 127.11.14.66:57711
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.14.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.67.keytab.
Entry for principal kudu/127.11.14.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.67.keytab.
I20260807 18:33:05.163146 11321 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkz56XN/build/debug/bin/kudu
/tmp/dist-test-taskkz56XN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127579900523-11321-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127579900523-11321-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127579900523-11321-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127579900523-11321-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.14.67:0
--local_ip_for_outbound_sockets=127.11.14.67
--webserver_interface=127.11.14.67
--webserver_port=0
--tserver_master_addrs=127.11.14.126:39415
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.67.keytab
--principal=kudu/127.11.14.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.14.84:43881
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127579900523-11321-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127579900523-11321-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127579900523-11321-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:33:05.397413 12307 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:33:05.397817 12307 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:33:05.397866 12307 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:33:05.403128 12307 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:33:05.403252 12307 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:33:05.403319 12307 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.14.67
I20260807 18:33:05.411106 12307 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.14.84:43881
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127579900523-11321-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127579900523-11321-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.67.keytab
--principal=kudu/127.11.14.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.14.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127579900523-11321-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.14.67
--webserver_port=0
--tserver_master_addrs=127.11.14.126:39415
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12307
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.14.67
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127579900523-11321-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:33:05.413038 12307 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:33:05.414490 12307 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:33:05.427814 12313 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:33:05.427999 12307 server_base.cc:1034] running on GCE node
W20260807 18:33:05.427906 12316 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:33:05.428076 12314 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:33:05.428802 12307 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:33:05.430500 12307 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:33:05.431762 12307 hybrid_clock.cc:648] HybridClock initialized: now 1786127585431736 us; error 32 us; skew 500 ppm
Aug 07 18:33:05 dist-test-slave-6k22 krb5kdc[11897](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127585, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:33:05.437569 12307 init.cc:377] Logged in from keytab as kudu/127.11.14.67@KRBTEST.COM (short username kudu)
I20260807 18:33:05.439040 12307 webserver.cc:460] Webserver started at http://127.11.14.67:35355/ using document root <none> and password file <none>
I20260807 18:33:05.439774 12307 fs_manager.cc:359] Metadata directory not provided
I20260807 18:33:05.439872 12307 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:33:05.440176 12307 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:33:05.443341 12307 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127579900523-11321-0/minicluster-data/ts-2/data/instance:
uuid: "21472425e61e41ef92c8b44b5d59cbf9"
format_stamp: "Formatted at 2026-08-07 18:33:05 on dist-test-slave-6k22"
server_key: "54b8fef4b1e5ce38abac39eb5c6c1511"
server_key_iv: "72cc5addf51850e3a8c6d36f4b5eddae"
server_key_version: "encryptionkey@0"
I20260807 18:33:05.444087 12307 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127579900523-11321-0/minicluster-data/ts-2/wal/instance:
uuid: "21472425e61e41ef92c8b44b5d59cbf9"
format_stamp: "Formatted at 2026-08-07 18:33:05 on dist-test-slave-6k22"
server_key: "54b8fef4b1e5ce38abac39eb5c6c1511"
server_key_iv: "72cc5addf51850e3a8c6d36f4b5eddae"
server_key_version: "encryptionkey@0"
I20260807 18:33:05.451067 12307 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.005s sys 0.000s
I20260807 18:33:05.454958 12323 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:33:05.458559 12307 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.000s
I20260807 18:33:05.458877 12307 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127579900523-11321-0/minicluster-data/ts-2/data,/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127579900523-11321-0/minicluster-data/ts-2/wal
uuid: "21472425e61e41ef92c8b44b5d59cbf9"
format_stamp: "Formatted at 2026-08-07 18:33:05 on dist-test-slave-6k22"
server_key: "54b8fef4b1e5ce38abac39eb5c6c1511"
server_key_iv: "72cc5addf51850e3a8c6d36f4b5eddae"
server_key_version: "encryptionkey@0"
I20260807 18:33:05.459204 12307 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127579900523-11321-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127579900523-11321-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127579900523-11321-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:33:05.488541 12307 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:33:05.497263 12307 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:33:05.497870 12307 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:33:05.500181 12307 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:33:05.502465 12307 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:33:05.502609 12307 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:33:05.502794 12307 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:33:05.502876 12307 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:33:05.544684 12307 rpc_server.cc:307] RPC server started. Bound to: 127.11.14.67:46697
I20260807 18:33:05.545089 12445 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.14.67:46697 every 8 connection(s)
I20260807 18:33:05.547082 12307 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127579900523-11321-0/minicluster-data/ts-2/data/info.pb
I20260807 18:33:05.547825 11321 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkz56XN/build/debug/bin/kudu as pid 12307
I20260807 18:33:05.548074 11321 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786127579900523-11321-0/minicluster-data/ts-2/wal/instance
I20260807 18:33:05.548787 11321 external_mini_cluster.cc:1428] Setting key 7e92d4de9bcfe412818613c176463f3b
Aug 07 18:33:05 dist-test-slave-6k22 krb5kdc[11897](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127585, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.67@KRBTEST.COM for kudu/127.11.14.126@KRBTEST.COM
I20260807 18:33:05.577944 12157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:05.550926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.67:38275 (local address 127.11.14.126:39415)
0807 18:33:05.551213 (+ 287us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:05.551219 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:05.553824 (+ 2605us) server_negotiation.cc:407] Connection header received
0807 18:33:05.556672 (+ 2848us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:05.556675 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:05.556772 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:05.556902 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:33:05.559524 (+ 2622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:05.560731 (+ 1207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:05.563055 (+ 2324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:05.563519 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:05.563799 (+ 280us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:05.570008 (+ 6209us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:33:05.570012 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:33:05.570014 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:33:05.570073 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:33:05.573483 (+ 3410us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:05.574607 (+ 1124us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:05.574609 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:05.574611 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:05.574759 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:05.575383 (+ 624us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:05.575394 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:05.575395 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:05.576132 (+ 737us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:33:05.576290 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:05.577400 (+ 1110us) server_negotiation.cc:299] Negotiation successful
0807 18:33:05.577667 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260807 18:33:05.580876 12448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:05.552215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:39415 (local address 127.11.14.67:38275)
0807 18:33:05.553454 (+ 1239us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:05.553624 (+ 170us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:05.556034 (+ 2410us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:05.557249 (+ 1215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:05.557258 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:05.558138 (+ 880us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:33:05.559318 (+ 1180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:05.559360 (+ 42us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:05.560924 (+ 1564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:05.560941 (+ 17us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:05.562859 (+ 1918us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:05.562871 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:05.563897 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:05.563908 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:05.564324 (+ 416us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:05.565992 (+ 1668us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:33:05.566031 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:33:05.569671 (+ 3640us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:33:05.573700 (+ 4029us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:33:05.573708 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:33:05.573750 (+ 42us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:33:05.574423 (+ 673us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:33:05.574873 (+ 450us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:33:05.574875 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:33:05.574877 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:33:05.575181 (+ 304us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:33:05.576331 (+ 1150us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:33:05.576354 (+ 23us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:33:05.576982 (+ 628us) client_negotiation.cc:769] Sending connection context
0807 18:33:05.577366 (+ 384us) client_negotiation.cc:240] Negotiation successful
0807 18:33:05.577994 (+ 628us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":689,"thread_start_us":332,"threads_started":1}
I20260807 18:33:05.583263 12446 heartbeater.cc:347] Connected to a master server at 127.11.14.126:39415
I20260807 18:33:05.583750 12446 heartbeater.cc:464] Registering TS with master...
I20260807 18:33:05.584798 12446 heartbeater.cc:511] Master 127.11.14.126:39415 requested a full tablet report, sending...
I20260807 18:33:05.586882 11945 ts_manager.cc:194] Registered new tserver with Master: 21472425e61e41ef92c8b44b5d59cbf9 (127.11.14.67:46697)
I20260807 18:33:05.588260 11945 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.14.67@KRBTEST.COM'} at 127.11.14.67:38275
I20260807 18:33:05.595026 11321 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 18:33:05.620790 12157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:05.603769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60082 (local address 127.11.14.126:39415)
0807 18:33:05.604145 (+ 376us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:05.604171 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:05.604875 (+ 704us) server_negotiation.cc:407] Connection header received
0807 18:33:05.605222 (+ 347us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:05.605226 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:05.605380 (+ 154us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:05.605571 (+ 191us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:33:05.607985 (+ 2414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:05.609036 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:05.610607 (+ 1571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:05.610997 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:05.611149 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:05.614293 (+ 3144us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:33:05.614297 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:33:05.614299 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:33:05.614349 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:33:05.617535 (+ 3186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:05.618290 (+ 755us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:05.618297 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:05.618301 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:05.618512 (+ 211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:05.619004 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:05.619009 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:05.619010 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:05.619439 (+ 429us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:33:05.619749 (+ 310us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:05.620138 (+ 389us) server_negotiation.cc:299] Negotiation successful
0807 18:33:05.620436 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104}
I20260807 18:33:05.632577 11945 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60082:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 18:33:05.639703 11945 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 18:33:05.675040 12461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:05.661180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.66:43123 (local address 127.0.0.1:60578)
0807 18:33:05.663899 (+ 2719us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:05.663925 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:05.664179 (+ 254us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:05.665388 (+ 1209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:05.665409 (+ 21us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:05.665493 (+ 84us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:33:05.665819 (+ 326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:05.665845 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:05.668453 (+ 2608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:05.668477 (+ 24us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:05.670983 (+ 2506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:05.671004 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:05.673808 (+ 2804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:05.673813 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:05.674050 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:05.674173 (+ 123us) client_negotiation.cc:769] Sending connection context
0807 18:33:05.674394 (+ 221us) client_negotiation.cc:240] Negotiation successful
0807 18:33:05.674637 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2589,"thread_start_us":128,"threads_started":1}
I20260807 18:33:05.675041 12460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:05.660894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.67:46697 (local address 127.0.0.1:38204)
0807 18:33:05.661915 (+ 1021us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:05.662059 (+ 144us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:05.662403 (+ 344us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:05.667012 (+ 4609us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:05.667016 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:05.667035 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:33:05.667226 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:05.667243 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:05.670225 (+ 2982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:05.670228 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:05.672103 (+ 1875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:05.672117 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:05.674199 (+ 2082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:05.674205 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:05.674501 (+ 296us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:05.674533 (+ 32us) client_negotiation.cc:769] Sending connection context
0807 18:33:05.674612 (+ 79us) client_negotiation.cc:240] Negotiation successful
0807 18:33:05.674717 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":463,"thread_start_us":210,"threads_started":1}
I20260807 18:33:05.676348 12462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:05.662781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38204 (local address 127.11.14.67:46697)
0807 18:33:05.666003 (+ 3222us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:05.666030 (+ 27us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:05.666078 (+ 48us) server_negotiation.cc:407] Connection header received
0807 18:33:05.666247 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:05.666264 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:05.666604 (+ 340us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:05.667615 (+ 1011us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:33:05.667944 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:05.669944 (+ 2000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:05.672397 (+ 2453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:05.673975 (+ 1578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:05.675205 (+ 1230us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:05.675708 (+ 503us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:05.675787 (+ 79us) server_negotiation.cc:299] Negotiation successful
0807 18:33:05.676023 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3020,"thread_start_us":222,"threads_started":1}
I20260807 18:33:05.677802 12463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:05.662761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60578 (local address 127.11.14.66:43123)
0807 18:33:05.664377 (+ 1616us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:05.664409 (+ 32us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:05.664457 (+ 48us) server_negotiation.cc:407] Connection header received
0807 18:33:05.664624 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:05.664644 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:05.665070 (+ 426us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:05.665423 (+ 353us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:33:05.666042 (+ 619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:05.668157 (+ 2115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:05.671300 (+ 3143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:05.673448 (+ 2148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:05.676948 (+ 3500us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:05.677287 (+ 339us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:05.677387 (+ 100us) server_negotiation.cc:299] Negotiation successful
0807 18:33:05.677536 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1365,"thread_start_us":230,"threads_started":1}
I20260807 18:33:05.679468 12465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:05.664885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57896 (local address 127.11.14.65:32989)
0807 18:33:05.668245 (+ 3360us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:05.668269 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:05.668323 (+ 54us) server_negotiation.cc:407] Connection header received
0807 18:33:05.668564 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:05.668572 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:05.668942 (+ 370us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:05.669298 (+ 356us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:33:05.670329 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:05.672824 (+ 2495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:05.675786 (+ 2962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:05.677545 (+ 1759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:05.677804 (+ 259us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:05.678190 (+ 386us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:05.678850 (+ 660us) server_negotiation.cc:299] Negotiation successful
0807 18:33:05.679160 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":929,"thread_start_us":286,"threads_started":1}
I20260807 18:33:05.680382 12464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:05.664611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.65:32989 (local address 127.0.0.1:57896)
0807 18:33:05.666423 (+ 1812us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:05.666461 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:05.666740 (+ 279us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:05.669585 (+ 2845us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:05.669602 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:05.669638 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:33:05.670059 (+ 421us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:05.670081 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:05.673039 (+ 2958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:05.673051 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:05.675436 (+ 2385us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:05.675445 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:05.677789 (+ 2344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:05.677804 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:05.678455 (+ 651us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:05.678493 (+ 38us) client_negotiation.cc:769] Sending connection context
0807 18:33:05.678702 (+ 209us) client_negotiation.cc:240] Negotiation successful
0807 18:33:05.678827 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1580,"thread_start_us":380,"threads_started":1}
I20260807 18:33:05.684360 12231 tablet_service.cc:1467] Processing CreateTablet for tablet 3d87752ccf4f4bba9fed99130198cbf3 (DEFAULT_TABLE table=test-table [id=03de0f850cc44c0e998b8c2ccb04a89f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:33:05.684110 12375 tablet_service.cc:1467] Processing CreateTablet for tablet 3d87752ccf4f4bba9fed99130198cbf3 (DEFAULT_TABLE table=test-table [id=03de0f850cc44c0e998b8c2ccb04a89f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:33:05.686585 12231 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3d87752ccf4f4bba9fed99130198cbf3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:33:05.685513 12082 tablet_service.cc:1467] Processing CreateTablet for tablet 3d87752ccf4f4bba9fed99130198cbf3 (DEFAULT_TABLE table=test-table [id=03de0f850cc44c0e998b8c2ccb04a89f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:33:05.687850 12082 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3d87752ccf4f4bba9fed99130198cbf3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:33:05.689095 12375 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3d87752ccf4f4bba9fed99130198cbf3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:33:05.705310 12466 tablet_bootstrap.cc:492] T 3d87752ccf4f4bba9fed99130198cbf3 P 1c688728860e473ba09b74079a76bff0: Bootstrap starting.
I20260807 18:33:05.705720 12467 tablet_bootstrap.cc:492] T 3d87752ccf4f4bba9fed99130198cbf3 P 0b40acc5583a4f11b6f01ecb91adda7f: Bootstrap starting.
I20260807 18:33:05.709827 12468 tablet_bootstrap.cc:492] T 3d87752ccf4f4bba9fed99130198cbf3 P 21472425e61e41ef92c8b44b5d59cbf9: Bootstrap starting.
I20260807 18:33:05.712720 12467 tablet_bootstrap.cc:654] T 3d87752ccf4f4bba9fed99130198cbf3 P 0b40acc5583a4f11b6f01ecb91adda7f: Neither blocks nor log segments found. Creating new log.
I20260807 18:33:05.715029 12466 tablet_bootstrap.cc:654] T 3d87752ccf4f4bba9fed99130198cbf3 P 1c688728860e473ba09b74079a76bff0: Neither blocks nor log segments found. Creating new log.
I20260807 18:33:05.716132 12467 log.cc:824] T 3d87752ccf4f4bba9fed99130198cbf3 P 0b40acc5583a4f11b6f01ecb91adda7f: Log is configured to *not* fsync() on all Append() calls
I20260807 18:33:05.716797 12468 tablet_bootstrap.cc:654] T 3d87752ccf4f4bba9fed99130198cbf3 P 21472425e61e41ef92c8b44b5d59cbf9: Neither blocks nor log segments found. Creating new log.
I20260807 18:33:05.719503 12468 log.cc:824] T 3d87752ccf4f4bba9fed99130198cbf3 P 21472425e61e41ef92c8b44b5d59cbf9: Log is configured to *not* fsync() on all Append() calls
I20260807 18:33:05.720414 12466 log.cc:824] T 3d87752ccf4f4bba9fed99130198cbf3 P 1c688728860e473ba09b74079a76bff0: Log is configured to *not* fsync() on all Append() calls
I20260807 18:33:05.720866 12467 tablet_bootstrap.cc:492] T 3d87752ccf4f4bba9fed99130198cbf3 P 0b40acc5583a4f11b6f01ecb91adda7f: No bootstrap required, opened a new log
I20260807 18:33:05.721267 12467 ts_tablet_manager.cc:1397] T 3d87752ccf4f4bba9fed99130198cbf3 P 0b40acc5583a4f11b6f01ecb91adda7f: Time spent bootstrapping tablet: real 0.016s user 0.006s sys 0.008s
I20260807 18:33:05.725235 12468 tablet_bootstrap.cc:492] T 3d87752ccf4f4bba9fed99130198cbf3 P 21472425e61e41ef92c8b44b5d59cbf9: No bootstrap required, opened a new log
I20260807 18:33:05.725742 12468 ts_tablet_manager.cc:1397] T 3d87752ccf4f4bba9fed99130198cbf3 P 21472425e61e41ef92c8b44b5d59cbf9: Time spent bootstrapping tablet: real 0.016s user 0.002s sys 0.011s
I20260807 18:33:05.727645 12466 tablet_bootstrap.cc:492] T 3d87752ccf4f4bba9fed99130198cbf3 P 1c688728860e473ba09b74079a76bff0: No bootstrap required, opened a new log
I20260807 18:33:05.727993 12466 ts_tablet_manager.cc:1397] T 3d87752ccf4f4bba9fed99130198cbf3 P 1c688728860e473ba09b74079a76bff0: Time spent bootstrapping tablet: real 0.025s user 0.012s sys 0.006s
I20260807 18:33:05.731036 12467 raft_consensus.cc:348] T 3d87752ccf4f4bba9fed99130198cbf3 P 0b40acc5583a4f11b6f01ecb91adda7f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21472425e61e41ef92c8b44b5d59cbf9" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 46697 } } peers { permanent_uuid: "0b40acc5583a4f11b6f01ecb91adda7f" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 43123 } } peers { permanent_uuid: "1c688728860e473ba09b74079a76bff0" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 32989 } }
I20260807 18:33:05.731482 12467 raft_consensus.cc:374] T 3d87752ccf4f4bba9fed99130198cbf3 P 0b40acc5583a4f11b6f01ecb91adda7f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:33:05.731542 12467 raft_consensus.cc:729] T 3d87752ccf4f4bba9fed99130198cbf3 P 0b40acc5583a4f11b6f01ecb91adda7f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b40acc5583a4f11b6f01ecb91adda7f, State: Initialized, Role: FOLLOWER
I20260807 18:33:05.732493 12467 consensus_queue.cc:260] T 3d87752ccf4f4bba9fed99130198cbf3 P 0b40acc5583a4f11b6f01ecb91adda7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21472425e61e41ef92c8b44b5d59cbf9" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 46697 } } peers { permanent_uuid: "0b40acc5583a4f11b6f01ecb91adda7f" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 43123 } } peers { permanent_uuid: "1c688728860e473ba09b74079a76bff0" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 32989 } }
I20260807 18:33:05.733902 12302 heartbeater.cc:503] Master 127.11.14.126:39415 was elected leader, sending a full tablet report...
I20260807 18:33:05.734683 12467 ts_tablet_manager.cc:1428] T 3d87752ccf4f4bba9fed99130198cbf3 P 0b40acc5583a4f11b6f01ecb91adda7f: Time spent starting tablet: real 0.013s user 0.013s sys 0.000s
I20260807 18:33:05.738107 12468 raft_consensus.cc:348] T 3d87752ccf4f4bba9fed99130198cbf3 P 21472425e61e41ef92c8b44b5d59cbf9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21472425e61e41ef92c8b44b5d59cbf9" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 46697 } } peers { permanent_uuid: "0b40acc5583a4f11b6f01ecb91adda7f" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 43123 } } peers { permanent_uuid: "1c688728860e473ba09b74079a76bff0" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 32989 } }
I20260807 18:33:05.738893 12468 raft_consensus.cc:374] T 3d87752ccf4f4bba9fed99130198cbf3 P 21472425e61e41ef92c8b44b5d59cbf9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:33:05.739185 12468 raft_consensus.cc:729] T 3d87752ccf4f4bba9fed99130198cbf3 P 21472425e61e41ef92c8b44b5d59cbf9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 21472425e61e41ef92c8b44b5d59cbf9, State: Initialized, Role: FOLLOWER
I20260807 18:33:05.740258 12468 consensus_queue.cc:260] T 3d87752ccf4f4bba9fed99130198cbf3 P 21472425e61e41ef92c8b44b5d59cbf9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21472425e61e41ef92c8b44b5d59cbf9" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 46697 } } peers { permanent_uuid: "0b40acc5583a4f11b6f01ecb91adda7f" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 43123 } } peers { permanent_uuid: "1c688728860e473ba09b74079a76bff0" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 32989 } }
I20260807 18:33:05.741201 12466 raft_consensus.cc:348] T 3d87752ccf4f4bba9fed99130198cbf3 P 1c688728860e473ba09b74079a76bff0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21472425e61e41ef92c8b44b5d59cbf9" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 46697 } } peers { permanent_uuid: "0b40acc5583a4f11b6f01ecb91adda7f" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 43123 } } peers { permanent_uuid: "1c688728860e473ba09b74079a76bff0" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 32989 } }
I20260807 18:33:05.742064 12466 raft_consensus.cc:374] T 3d87752ccf4f4bba9fed99130198cbf3 P 1c688728860e473ba09b74079a76bff0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:33:05.742786 12468 ts_tablet_manager.cc:1428] T 3d87752ccf4f4bba9fed99130198cbf3 P 21472425e61e41ef92c8b44b5d59cbf9: Time spent starting tablet: real 0.017s user 0.013s sys 0.004s
I20260807 18:33:05.742784 12155 heartbeater.cc:503] Master 127.11.14.126:39415 was elected leader, sending a full tablet report...
I20260807 18:33:05.742318 12466 raft_consensus.cc:729] T 3d87752ccf4f4bba9fed99130198cbf3 P 1c688728860e473ba09b74079a76bff0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c688728860e473ba09b74079a76bff0, State: Initialized, Role: FOLLOWER
I20260807 18:33:05.743453 12446 heartbeater.cc:503] Master 127.11.14.126:39415 was elected leader, sending a full tablet report...
I20260807 18:33:05.744500 12466 consensus_queue.cc:260] T 3d87752ccf4f4bba9fed99130198cbf3 P 1c688728860e473ba09b74079a76bff0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21472425e61e41ef92c8b44b5d59cbf9" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 46697 } } peers { permanent_uuid: "0b40acc5583a4f11b6f01ecb91adda7f" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 43123 } } peers { permanent_uuid: "1c688728860e473ba09b74079a76bff0" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 32989 } }
I20260807 18:33:05.746901 12466 ts_tablet_manager.cc:1428] T 3d87752ccf4f4bba9fed99130198cbf3 P 1c688728860e473ba09b74079a76bff0: Time spent starting tablet: real 0.019s user 0.008s sys 0.009s
W20260807 18:33:05.799443 12447 tablet.cc:2369] T 3d87752ccf4f4bba9fed99130198cbf3 P 21472425e61e41ef92c8b44b5d59cbf9: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 18:33:05.852185 12303 tablet.cc:2369] T 3d87752ccf4f4bba9fed99130198cbf3 P 0b40acc5583a4f11b6f01ecb91adda7f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:33:05.949445 12472 raft_consensus.cc:482] T 3d87752ccf4f4bba9fed99130198cbf3 P 0b40acc5583a4f11b6f01ecb91adda7f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:33:05.950067 12472 raft_consensus.cc:504] T 3d87752ccf4f4bba9fed99130198cbf3 P 0b40acc5583a4f11b6f01ecb91adda7f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21472425e61e41ef92c8b44b5d59cbf9" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 46697 } } peers { permanent_uuid: "0b40acc5583a4f11b6f01ecb91adda7f" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 43123 } } peers { permanent_uuid: "1c688728860e473ba09b74079a76bff0" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 32989 } }
I20260807 18:33:05.952507 12472 leader_election.cc:288] T 3d87752ccf4f4bba9fed99130198cbf3 P 0b40acc5583a4f11b6f01ecb91adda7f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 21472425e61e41ef92c8b44b5d59cbf9 (127.11.14.67:46697), 1c688728860e473ba09b74079a76bff0 (127.11.14.65:32989)
W20260807 18:33:05.956548 12156 tablet.cc:2369] T 3d87752ccf4f4bba9fed99130198cbf3 P 1c688728860e473ba09b74079a76bff0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:33:05.962662 12465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:05.953395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.66:60605 (local address 127.11.14.65:32989)
0807 18:33:05.954068 (+ 673us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:05.954076 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:05.954130 (+ 54us) server_negotiation.cc:407] Connection header received
0807 18:33:05.954337 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:05.954342 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:05.954465 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:05.954649 (+ 184us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:33:05.955649 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:05.956937 (+ 1288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:05.960170 (+ 3233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:05.961498 (+ 1328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:05.961717 (+ 219us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:05.961880 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:05.962249 (+ 369us) server_negotiation.cc:299] Negotiation successful
0807 18:33:05.962439 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":364}
I20260807 18:33:05.963078 12462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:05.953390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.66:48141 (local address 127.11.14.67:46697)
0807 18:33:05.953702 (+ 312us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:05.953707 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:05.954686 (+ 979us) server_negotiation.cc:407] Connection header received
0807 18:33:05.955102 (+ 416us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:05.955107 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:05.955246 (+ 139us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:05.955469 (+ 223us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:33:05.956389 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:05.957525 (+ 1136us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:05.960707 (+ 3182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:05.961927 (+ 1220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:05.962129 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:05.962274 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:05.962669 (+ 395us) server_negotiation.cc:299] Negotiation successful
0807 18:33:05.962962 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126}
I20260807 18:33:05.963121 12475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:05.953068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.67:46697 (local address 127.11.14.66:48141)
0807 18:33:05.954479 (+ 1411us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:05.954507 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:05.954800 (+ 293us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:05.955508 (+ 708us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:05.955515 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:05.955625 (+ 110us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:33:05.956140 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:05.956174 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:05.957753 (+ 1579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:05.957757 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:05.960435 (+ 2678us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:05.960458 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:05.962160 (+ 1702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:05.962168 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:05.962441 (+ 273us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:05.962462 (+ 21us) client_negotiation.cc:769] Sending connection context
0807 18:33:05.962594 (+ 132us) client_negotiation.cc:240] Negotiation successful
0807 18:33:05.962702 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1087,"mutex_wait_us":45,"spinlock_wait_cycles":2304,"thread_start_us":240,"threads_started":1}
I20260807 18:33:05.963157 12476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:05.953068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.65:32989 (local address 127.11.14.66:60605)
0807 18:33:05.953829 (+ 761us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:05.953871 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:05.954091 (+ 220us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:05.954690 (+ 599us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:05.954694 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:05.954796 (+ 102us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:33:05.955315 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:05.955329 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:05.957123 (+ 1794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:05.957141 (+ 18us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:05.959848 (+ 2707us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:05.959871 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:05.961693 (+ 1822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:05.961697 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:05.961931 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:05.961984 (+ 53us) client_negotiation.cc:769] Sending connection context
0807 18:33:05.962221 (+ 237us) client_negotiation.cc:240] Negotiation successful
0807 18:33:05.962395 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":604,"spinlock_wait_cycles":44032,"thread_start_us":323,"threads_started":1}
I20260807 18:33:05.964751 12398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d87752ccf4f4bba9fed99130198cbf3" candidate_uuid: "0b40acc5583a4f11b6f01ecb91adda7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21472425e61e41ef92c8b44b5d59cbf9" is_pre_election: true
I20260807 18:33:05.964700 12103 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d87752ccf4f4bba9fed99130198cbf3" candidate_uuid: "0b40acc5583a4f11b6f01ecb91adda7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c688728860e473ba09b74079a76bff0" is_pre_election: true
I20260807 18:33:05.965613 12103 raft_consensus.cc:2454] T 3d87752ccf4f4bba9fed99130198cbf3 P 1c688728860e473ba09b74079a76bff0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0b40acc5583a4f11b6f01ecb91adda7f in term 0.
I20260807 18:33:05.965626 12398 raft_consensus.cc:2454] T 3d87752ccf4f4bba9fed99130198cbf3 P 21472425e61e41ef92c8b44b5d59cbf9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0b40acc5583a4f11b6f01ecb91adda7f in term 0.
I20260807 18:33:05.966682 12182 leader_election.cc:302] T 3d87752ccf4f4bba9fed99130198cbf3 P 0b40acc5583a4f11b6f01ecb91adda7f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 0b40acc5583a4f11b6f01ecb91adda7f, 1c688728860e473ba09b74079a76bff0, 21472425e61e41ef92c8b44b5d59cbf9; no voters:
I20260807 18:33:05.967233 12472 raft_consensus.cc:2781] T 3d87752ccf4f4bba9fed99130198cbf3 P 0b40acc5583a4f11b6f01ecb91adda7f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:33:05.967408 12472 raft_consensus.cc:482] T 3d87752ccf4f4bba9fed99130198cbf3 P 0b40acc5583a4f11b6f01ecb91adda7f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:33:05.967549 12472 raft_consensus.cc:3037] T 3d87752ccf4f4bba9fed99130198cbf3 P 0b40acc5583a4f11b6f01ecb91adda7f [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:33:05.979359 12472 raft_consensus.cc:504] T 3d87752ccf4f4bba9fed99130198cbf3 P 0b40acc5583a4f11b6f01ecb91adda7f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21472425e61e41ef92c8b44b5d59cbf9" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 46697 } } peers { permanent_uuid: "0b40acc5583a4f11b6f01ecb91adda7f" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 43123 } } peers { permanent_uuid: "1c688728860e473ba09b74079a76bff0" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 32989 } }
I20260807 18:33:05.980212 12472 leader_election.cc:288] T 3d87752ccf4f4bba9fed99130198cbf3 P 0b40acc5583a4f11b6f01ecb91adda7f [CANDIDATE]: Term 1 election: Requested vote from peers 21472425e61e41ef92c8b44b5d59cbf9 (127.11.14.67:46697), 1c688728860e473ba09b74079a76bff0 (127.11.14.65:32989)
I20260807 18:33:05.980845 12398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d87752ccf4f4bba9fed99130198cbf3" candidate_uuid: "0b40acc5583a4f11b6f01ecb91adda7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21472425e61e41ef92c8b44b5d59cbf9"
I20260807 18:33:05.981216 12103 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d87752ccf4f4bba9fed99130198cbf3" candidate_uuid: "0b40acc5583a4f11b6f01ecb91adda7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c688728860e473ba09b74079a76bff0"
I20260807 18:33:05.981137 12398 raft_consensus.cc:3037] T 3d87752ccf4f4bba9fed99130198cbf3 P 21472425e61e41ef92c8b44b5d59cbf9 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:33:05.981446 12103 raft_consensus.cc:3037] T 3d87752ccf4f4bba9fed99130198cbf3 P 1c688728860e473ba09b74079a76bff0 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:33:05.983625 12103 raft_consensus.cc:2454] T 3d87752ccf4f4bba9fed99130198cbf3 P 1c688728860e473ba09b74079a76bff0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0b40acc5583a4f11b6f01ecb91adda7f in term 1.
I20260807 18:33:05.984128 12182 leader_election.cc:302] T 3d87752ccf4f4bba9fed99130198cbf3 P 0b40acc5583a4f11b6f01ecb91adda7f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b40acc5583a4f11b6f01ecb91adda7f, 1c688728860e473ba09b74079a76bff0; no voters:
I20260807 18:33:05.984421 12472 raft_consensus.cc:2781] T 3d87752ccf4f4bba9fed99130198cbf3 P 0b40acc5583a4f11b6f01ecb91adda7f [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:33:05.984859 12472 raft_consensus.cc:686] T 3d87752ccf4f4bba9fed99130198cbf3 P 0b40acc5583a4f11b6f01ecb91adda7f [term 1 LEADER]: Becoming Leader. State: Replica: 0b40acc5583a4f11b6f01ecb91adda7f, State: Running, Role: LEADER
I20260807 18:33:05.985649 12398 raft_consensus.cc:2454] T 3d87752ccf4f4bba9fed99130198cbf3 P 21472425e61e41ef92c8b44b5d59cbf9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0b40acc5583a4f11b6f01ecb91adda7f in term 1.
I20260807 18:33:05.985805 12472 consensus_queue.cc:237] T 3d87752ccf4f4bba9fed99130198cbf3 P 0b40acc5583a4f11b6f01ecb91adda7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21472425e61e41ef92c8b44b5d59cbf9" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 46697 } } peers { permanent_uuid: "0b40acc5583a4f11b6f01ecb91adda7f" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 43123 } } peers { permanent_uuid: "1c688728860e473ba09b74079a76bff0" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 32989 } }
I20260807 18:33:05.992192 11944 catalog_manager.cc:5495] T 3d87752ccf4f4bba9fed99130198cbf3 P 0b40acc5583a4f11b6f01ecb91adda7f reported cstate change: term changed from 0 to 1, leader changed from <none> to 0b40acc5583a4f11b6f01ecb91adda7f (127.11.14.66). New cstate: current_term: 1 leader_uuid: "0b40acc5583a4f11b6f01ecb91adda7f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21472425e61e41ef92c8b44b5d59cbf9" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 46697 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0b40acc5583a4f11b6f01ecb91adda7f" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 43123 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1c688728860e473ba09b74079a76bff0" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 32989 } health_report { overall_health: UNKNOWN } } }
Aug 07 18:33:06 dist-test-slave-6k22 krb5kdc[11897](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127583, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.14.65@KRBTEST.COM
I20260807 18:33:06.068084 12465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:06.049088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57912 (local address 127.11.14.65:32989)
0807 18:33:06.049471 (+ 383us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:06.049479 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:06.049854 (+ 375us) server_negotiation.cc:407] Connection header received
0807 18:33:06.050068 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:06.050071 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:06.050150 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:06.050357 (+ 207us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:33:06.051788 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:06.052658 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:06.054813 (+ 2155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:06.055244 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:06.055491 (+ 247us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:06.060926 (+ 5435us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:33:06.060946 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:33:06.060974 (+ 28us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:33:06.061030 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:33:06.064789 (+ 3759us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:06.065699 (+ 910us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:06.065714 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:06.065715 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:06.065871 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:06.066407 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:06.066410 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:06.066411 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:06.066930 (+ 519us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:33:06.067263 (+ 333us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:06.067525 (+ 262us) server_negotiation.cc:299] Negotiation successful
0807 18:33:06.067854 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109}
Aug 07 18:33:06 dist-test-slave-6k22 krb5kdc[11897](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127586, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
W20260807 18:33:06.198385 11321 mini_kdc.cc:317] Time spent kinit for test-user: real 0.125s user 0.003s sys 0.006s
Aug 07 18:33:06 dist-test-slave-6k22 krb5kdc[11897](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127586, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.11.14.65@KRBTEST.COM
I20260807 18:33:06.210994 12465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:06.200592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57922 (local address 127.11.14.65:32989)
0807 18:33:06.200783 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:06.200788 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:06.200808 (+ 20us) server_negotiation.cc:407] Connection header received
0807 18:33:06.200889 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:06.200892 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:06.200950 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:06.201065 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:33:06.201871 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:06.202895 (+ 1024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:06.203918 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:06.204355 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:06.204467 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:06.207527 (+ 3060us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:33:06.207533 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:33:06.207538 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:33:06.207600 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:33:06.209320 (+ 1720us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:06.209760 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:06.209762 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:06.209764 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:06.209847 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:06.210093 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:06.210096 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:06.210098 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:06.210275 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:33:06.210383 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:06.210638 (+ 255us) server_negotiation.cc:299] Negotiation successful
0807 18:33:06.210789 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
Aug 07 18:33:06 dist-test-slave-6k22 krb5kdc[11897](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127586, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 07 18:33:06 dist-test-slave-6k22 krb5kdc[11897](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127586, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.14.65@KRBTEST.COM
I20260807 18:33:06.314276 12465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:06.304965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57930 (local address 127.11.14.65:32989)
0807 18:33:06.305130 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:06.305137 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:06.305351 (+ 214us) server_negotiation.cc:407] Connection header received
0807 18:33:06.305536 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:06.305540 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:06.305598 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:06.305696 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:33:06.306464 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:06.306981 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:06.307923 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:06.308151 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:06.308265 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:06.310890 (+ 2625us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:33:06.310893 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:33:06.310895 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:33:06.310920 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:33:06.312466 (+ 1546us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:06.312896 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:06.312899 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:06.312900 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:06.312979 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:06.313264 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:06.313268 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:06.313269 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:06.313529 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:33:06.313686 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:06.313937 (+ 251us) server_negotiation.cc:299] Negotiation successful
0807 18:33:06.314044 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
W20260807 18:33:06.316323 12058 tablet_service.cc:2908] Rejecting scan request for tablet 3d87752ccf4f4bba9fed99130198cbf3: Uninitialized: clean time has not yet been initialized
I20260807 18:33:06.319272 11321 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkz56XN/build/debug/bin/kudu with pid 12010
I20260807 18:33:06.328685 11321 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkz56XN/build/debug/bin/kudu with pid 12162
I20260807 18:33:06.337486 11321 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkz56XN/build/debug/bin/kudu with pid 12307
I20260807 18:33:06.345489 11321 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkz56XN/build/debug/bin/kudu with pid 11910
2026-08-07T18:33:06Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (3857 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127579900523-11321-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:33:07 dist-test-slave-6k22 krb5kdc[12492](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:33:07 dist-test-slave-6k22 krb5kdc[12492](info): set up 2 sockets
Aug 07 18:33:07 dist-test-slave-6k22 krb5kdc[12492](info): commencing operation
krb5kdc: starting...
W20260807 18:33:08.154697 11321 mini_kdc.cc:121] Time spent starting KDC: real 1.784s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127579900523-11321-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127579900523-11321-0/krb5kdc/test-user.keytab.
Aug 07 18:33:08 dist-test-slave-6k22 krb5kdc[12492](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127588, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:33:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:33:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.14.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.14.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.14.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.126.keytab.
Entry for principal kudu/127.11.14.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.126.keytab.
I20260807 18:33:08.288821 11321 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkz56XN/build/debug/bin/kudu
/tmp/dist-test-taskkz56XN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127579900523-11321-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127579900523-11321-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127579900523-11321-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127579900523-11321-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.14.126:34397
--webserver_interface=127.11.14.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.126.keytab
--principal=kudu/127.11.14.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.14.84:46775
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.14.126:34397
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127579900523-11321-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127579900523-11321-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127579900523-11321-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:33:08.404242 12505 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:33:08.404574 12505 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:33:08.404675 12505 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:33:08.408792 12505 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:33:08.408890 12505 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260807 18:33:08.408921 12505 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:33:08.408990 12505 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:33:08.409045 12505 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:33:08.409085 12505 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:33:08.414553 12505 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.14.84:46775
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127579900523-11321-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127579900523-11321-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.14.126:34397
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.126.keytab
--principal=kudu/127.11.14.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.14.126:34397
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127579900523-11321-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.14.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12505
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127579900523-11321-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:33:08.415818 12505 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:33:08.416965 12505 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:33:08.425853 12515 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:33:08.425853 12511 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:33:08.425881 12512 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:33:08.426288 12505 server_base.cc:1034] running on GCE node
I20260807 18:33:08.426895 12505 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:33:08.428122 12505 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:33:08.429317 12505 hybrid_clock.cc:648] HybridClock initialized: now 1786127588429298 us; error 31 us; skew 500 ppm
Aug 07 18:33:08 dist-test-slave-6k22 krb5kdc[12492](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127588, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:33:08.432097 12505 init.cc:377] Logged in from keytab as kudu/127.11.14.126@KRBTEST.COM (short username kudu)
I20260807 18:33:08.432997 12505 webserver.cc:460] Webserver started at http://127.11.14.126:36579/ using document root <none> and password file <none>
I20260807 18:33:08.433594 12505 fs_manager.cc:359] Metadata directory not provided
I20260807 18:33:08.433688 12505 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:33:08.433954 12505 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:33:08.436218 12505 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127579900523-11321-0/minicluster-data/master-0/data/instance:
uuid: "a4c134b845384a56aa320c0f437b3512"
format_stamp: "Formatted at 2026-08-07 18:33:08 on dist-test-slave-6k22"
server_key: "c1b79a5cfa216058dde5a7e88f41dabf"
server_key_iv: "1cadb9dcb78711f992e2f3ee0a385f20"
server_key_version: "encryptionkey@0"
I20260807 18:33:08.436820 12505 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127579900523-11321-0/minicluster-data/master-0/wal/instance:
uuid: "a4c134b845384a56aa320c0f437b3512"
format_stamp: "Formatted at 2026-08-07 18:33:08 on dist-test-slave-6k22"
server_key: "c1b79a5cfa216058dde5a7e88f41dabf"
server_key_iv: "1cadb9dcb78711f992e2f3ee0a385f20"
server_key_version: "encryptionkey@0"
I20260807 18:33:08.440721 12505 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260807 18:33:08.443135 12522 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:33:08.444442 12505 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260807 18:33:08.444585 12505 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127579900523-11321-0/minicluster-data/master-0/data,/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127579900523-11321-0/minicluster-data/master-0/wal
uuid: "a4c134b845384a56aa320c0f437b3512"
format_stamp: "Formatted at 2026-08-07 18:33:08 on dist-test-slave-6k22"
server_key: "c1b79a5cfa216058dde5a7e88f41dabf"
server_key_iv: "1cadb9dcb78711f992e2f3ee0a385f20"
server_key_version: "encryptionkey@0"
I20260807 18:33:08.444700 12505 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127579900523-11321-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127579900523-11321-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127579900523-11321-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:33:08.465251 12505 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:33:08.468266 12505 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:33:08.468487 12505 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:33:08.476871 12505 rpc_server.cc:307] RPC server started. Bound to: 127.11.14.126:34397
I20260807 18:33:08.476889 12592 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.14.126:34397 every 8 connection(s)
I20260807 18:33:08.477874 12505 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127579900523-11321-0/minicluster-data/master-0/data/info.pb
I20260807 18:33:08.480675 12593 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:33:08.485810 11321 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkz56XN/build/debug/bin/kudu as pid 12505
I20260807 18:33:08.485941 11321 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127579900523-11321-0/minicluster-data/master-0/wal/instance
I20260807 18:33:08.486199 11321 external_mini_cluster.cc:1428] Setting key eb9db076d00b4a72f7cf8dc2a56bf095
I20260807 18:33:08.486615 12593 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a4c134b845384a56aa320c0f437b3512: Bootstrap starting.
I20260807 18:33:08.489275 12593 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a4c134b845384a56aa320c0f437b3512: Neither blocks nor log segments found. Creating new log.
I20260807 18:33:08.490046 12593 log.cc:824] T 00000000000000000000000000000000 P a4c134b845384a56aa320c0f437b3512: Log is configured to *not* fsync() on all Append() calls
I20260807 18:33:08.492065 12593 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a4c134b845384a56aa320c0f437b3512: No bootstrap required, opened a new log
Aug 07 18:33:08 dist-test-slave-6k22 krb5kdc[12492](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127588, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.14.126@KRBTEST.COM
I20260807 18:33:08.496066 12593 raft_consensus.cc:348] T 00000000000000000000000000000000 P a4c134b845384a56aa320c0f437b3512 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4c134b845384a56aa320c0f437b3512" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 34397 } }
I20260807 18:33:08.496325 12593 raft_consensus.cc:374] T 00000000000000000000000000000000 P a4c134b845384a56aa320c0f437b3512 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:33:08.496388 12593 raft_consensus.cc:729] T 00000000000000000000000000000000 P a4c134b845384a56aa320c0f437b3512 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4c134b845384a56aa320c0f437b3512, State: Initialized, Role: FOLLOWER
I20260807 18:33:08.496878 12593 consensus_queue.cc:260] T 00000000000000000000000000000000 P a4c134b845384a56aa320c0f437b3512 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4c134b845384a56aa320c0f437b3512" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 34397 } }
I20260807 18:33:08.497037 12593 raft_consensus.cc:388] T 00000000000000000000000000000000 P a4c134b845384a56aa320c0f437b3512 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:33:08.497115 12593 raft_consensus.cc:482] T 00000000000000000000000000000000 P a4c134b845384a56aa320c0f437b3512 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:33:08.497255 12593 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a4c134b845384a56aa320c0f437b3512 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:33:08.498488 12593 raft_consensus.cc:504] T 00000000000000000000000000000000 P a4c134b845384a56aa320c0f437b3512 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4c134b845384a56aa320c0f437b3512" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 34397 } }
I20260807 18:33:08.498884 12593 leader_election.cc:302] T 00000000000000000000000000000000 P a4c134b845384a56aa320c0f437b3512 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a4c134b845384a56aa320c0f437b3512; no voters:
I20260807 18:33:08.499262 12593 leader_election.cc:288] T 00000000000000000000000000000000 P a4c134b845384a56aa320c0f437b3512 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:33:08.499414 12600 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a4c134b845384a56aa320c0f437b3512 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:33:08.499673 12600 raft_consensus.cc:686] T 00000000000000000000000000000000 P a4c134b845384a56aa320c0f437b3512 [term 1 LEADER]: Becoming Leader. State: Replica: a4c134b845384a56aa320c0f437b3512, State: Running, Role: LEADER
I20260807 18:33:08.500042 12600 consensus_queue.cc:237] T 00000000000000000000000000000000 P a4c134b845384a56aa320c0f437b3512 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4c134b845384a56aa320c0f437b3512" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 34397 } }
I20260807 18:33:08.500737 12593 sys_catalog.cc:564] T 00000000000000000000000000000000 P a4c134b845384a56aa320c0f437b3512 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:33:08.501852 12602 sys_catalog.cc:455] T 00000000000000000000000000000000 P a4c134b845384a56aa320c0f437b3512 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a4c134b845384a56aa320c0f437b3512. Latest consensus state: current_term: 1 leader_uuid: "a4c134b845384a56aa320c0f437b3512" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4c134b845384a56aa320c0f437b3512" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 34397 } } }
I20260807 18:33:08.501979 12602 sys_catalog.cc:458] T 00000000000000000000000000000000 P a4c134b845384a56aa320c0f437b3512 [sys.catalog]: This master's current role is: LEADER
I20260807 18:33:08.502071 12597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:08.487761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53634 (local address 127.11.14.126:34397)
0807 18:33:08.488244 (+ 483us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:08.488261 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:08.488300 (+ 39us) server_negotiation.cc:407] Connection header received
0807 18:33:08.488952 (+ 652us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:08.488973 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:08.489312 (+ 339us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:08.489717 (+ 405us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:33:08.490574 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:08.491640 (+ 1066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:08.492578 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:08.493063 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:08.493255 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:08.496070 (+ 2815us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:33:08.496075 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:33:08.496090 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:33:08.496118 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:33:08.498284 (+ 2166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:08.498815 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:08.498821 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:08.498827 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:08.498978 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:08.499343 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:08.499348 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:08.499353 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:08.499890 (+ 537us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:33:08.500087 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:08.500708 (+ 621us) server_negotiation.cc:299] Negotiation successful
0807 18:33:08.501033 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":336,"thread_start_us":158,"threads_started":1}
I20260807 18:33:08.502564 12601 sys_catalog.cc:455] T 00000000000000000000000000000000 P a4c134b845384a56aa320c0f437b3512 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a4c134b845384a56aa320c0f437b3512" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4c134b845384a56aa320c0f437b3512" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 34397 } } }
I20260807 18:33:08.502719 12601 sys_catalog.cc:458] T 00000000000000000000000000000000 P a4c134b845384a56aa320c0f437b3512 [sys.catalog]: This master's current role is: LEADER
W20260807 18:33:08.505792 12615 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a4c134b845384a56aa320c0f437b3512: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 18:33:08.505883 12615 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 18:33:08.505970 12609 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:33:08.506645 12609 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:33:08.513236 12609 catalog_manager.cc:1347] Generated new cluster ID: fef6a45e6b754ab494b3082baa7d6b3c
I20260807 18:33:08.513342 12609 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:33:08.528280 12609 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:33:08.529513 12609 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:33:08.549367 12609 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a4c134b845384a56aa320c0f437b3512: Generated new TSK 0
I20260807 18:33:08.550256 12609 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.14.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.14.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.14.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.65.keytab.
Entry for principal kudu/127.11.14.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.65.keytab.
I20260807 18:33:08.583498 11321 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkz56XN/build/debug/bin/kudu
/tmp/dist-test-taskkz56XN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127579900523-11321-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127579900523-11321-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127579900523-11321-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127579900523-11321-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.14.65:0
--local_ip_for_outbound_sockets=127.11.14.65
--webserver_interface=127.11.14.65
--webserver_port=0
--tserver_master_addrs=127.11.14.126:34397
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.65.keytab
--principal=kudu/127.11.14.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.14.84:46775
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127579900523-11321-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127579900523-11321-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127579900523-11321-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:33:08.724594 12621 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:33:08.724918 12621 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:33:08.724999 12621 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:33:08.729031 12621 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:33:08.729120 12621 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:33:08.729194 12621 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 18:33:08.729271 12621 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.14.65
I20260807 18:33:08.734740 12621 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.14.84:46775
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127579900523-11321-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127579900523-11321-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.65.keytab
--principal=kudu/127.11.14.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.14.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127579900523-11321-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.14.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.11.14.126:34397
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12621
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.14.65
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127579900523-11321-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:33:08.735879 12621 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:33:08.736976 12621 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:33:08.747180 12629 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:33:08.747161 12626 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:33:08.747161 12627 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:33:08.747437 12621 server_base.cc:1034] running on GCE node
I20260807 18:33:08.748008 12621 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:33:08.748708 12621 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:33:08.749902 12621 hybrid_clock.cc:648] HybridClock initialized: now 1786127588749870 us; error 48 us; skew 500 ppm
Aug 07 18:33:08 dist-test-slave-6k22 krb5kdc[12492](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127588, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:33:08.753172 12621 init.cc:377] Logged in from keytab as kudu/127.11.14.65@KRBTEST.COM (short username kudu)
I20260807 18:33:08.753979 12621 webserver.cc:460] Webserver started at http://127.11.14.65:42175/ using document root <none> and password file <none>
I20260807 18:33:08.754495 12621 fs_manager.cc:359] Metadata directory not provided
I20260807 18:33:08.754554 12621 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:33:08.754724 12621 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:33:08.756940 12621 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127579900523-11321-0/minicluster-data/ts-0/data/instance:
uuid: "62c0ffbc8d8d46fdb61ff8f094e1bb68"
format_stamp: "Formatted at 2026-08-07 18:33:08 on dist-test-slave-6k22"
server_key: "da252c87aa56760634a178835d5e9e1b"
server_key_iv: "3a213de588ce0bde821912a9828a9ae2"
server_key_version: "encryptionkey@0"
I20260807 18:33:08.757444 12621 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127579900523-11321-0/minicluster-data/ts-0/wal/instance:
uuid: "62c0ffbc8d8d46fdb61ff8f094e1bb68"
format_stamp: "Formatted at 2026-08-07 18:33:08 on dist-test-slave-6k22"
server_key: "da252c87aa56760634a178835d5e9e1b"
server_key_iv: "3a213de588ce0bde821912a9828a9ae2"
server_key_version: "encryptionkey@0"
I20260807 18:33:08.761341 12621 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260807 18:33:08.764201 12636 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:33:08.765501 12621 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:33:08.765606 12621 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127579900523-11321-0/minicluster-data/ts-0/data,/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127579900523-11321-0/minicluster-data/ts-0/wal
uuid: "62c0ffbc8d8d46fdb61ff8f094e1bb68"
format_stamp: "Formatted at 2026-08-07 18:33:08 on dist-test-slave-6k22"
server_key: "da252c87aa56760634a178835d5e9e1b"
server_key_iv: "3a213de588ce0bde821912a9828a9ae2"
server_key_version: "encryptionkey@0"
I20260807 18:33:08.765690 12621 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127579900523-11321-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127579900523-11321-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127579900523-11321-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:33:08.776418 12621 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:33:08.779206 12621 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:33:08.779362 12621 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:33:08.780994 12621 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:33:08.781045 12621 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:33:08.781106 12621 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:33:08.781136 12621 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 18:33:08 dist-test-slave-6k22 krb5kdc[12492](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127588, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.65@KRBTEST.COM for kudu/127.11.14.126@KRBTEST.COM
I20260807 18:33:08.796948 12597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:08.781720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.65:52011 (local address 127.11.14.126:34397)
0807 18:33:08.781883 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:08.781889 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:08.783496 (+ 1607us) server_negotiation.cc:407] Connection header received
0807 18:33:08.784854 (+ 1358us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:08.784858 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:08.784914 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:08.785020 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:33:08.786437 (+ 1417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:08.786984 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:08.788438 (+ 1454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:08.788781 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:08.788911 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:08.792464 (+ 3553us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:33:08.792467 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:33:08.792468 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:33:08.792492 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:33:08.794355 (+ 1863us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:08.794936 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:08.794940 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:08.794942 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:08.795031 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:08.795514 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:08.795517 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:08.795519 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:08.795881 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:33:08.796036 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:08.796535 (+ 499us) server_negotiation.cc:299] Negotiation successful
0807 18:33:08.796706 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260807 18:33:08.797891 12647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:08.782692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:34397 (local address 127.11.14.65:52011)
0807 18:33:08.783292 (+ 600us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:08.783356 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:08.784632 (+ 1276us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:08.785198 (+ 566us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:08.785208 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:08.785620 (+ 412us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:33:08.786264 (+ 644us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:08.786282 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:08.787129 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:08.787133 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:08.788300 (+ 1167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:08.788310 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:08.788949 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:08.788953 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:08.789138 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:08.789852 (+ 714us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:33:08.789879 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:33:08.792267 (+ 2388us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:33:08.794474 (+ 2207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:33:08.794480 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:33:08.794529 (+ 49us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:33:08.794816 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:33:08.795128 (+ 312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:33:08.795132 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:33:08.795134 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:33:08.795340 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:33:08.796019 (+ 679us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:33:08.796024 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:33:08.796255 (+ 231us) client_negotiation.cc:769] Sending connection context
0807 18:33:08.796409 (+ 154us) client_negotiation.cc:240] Negotiation successful
0807 18:33:08.796713 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":372,"thread_start_us":230,"threads_started":1}
I20260807 18:33:08.800071 12621 rpc_server.cc:307] RPC server started. Bound to: 127.11.14.65:44813
I20260807 18:33:08.800730 12768 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.14.65:44813 every 8 connection(s)
I20260807 18:33:08.801306 12621 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127579900523-11321-0/minicluster-data/ts-0/data/info.pb
I20260807 18:33:08.803035 12773 heartbeater.cc:347] Connected to a master server at 127.11.14.126:34397
I20260807 18:33:08.803205 12773 heartbeater.cc:464] Registering TS with master...
I20260807 18:33:08.803800 12773 heartbeater.cc:511] Master 127.11.14.126:34397 requested a full tablet report, sending...
I20260807 18:33:08.805470 12543 ts_manager.cc:194] Registered new tserver with Master: 62c0ffbc8d8d46fdb61ff8f094e1bb68 (127.11.14.65:44813)
I20260807 18:33:08.807073 12543 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.14.65@KRBTEST.COM'} at 127.11.14.65:52011
I20260807 18:33:08.810454 12597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:08.802747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.65:35579 (local address 127.11.14.126:34397)
0807 18:33:08.802904 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:08.802908 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:08.803177 (+ 269us) server_negotiation.cc:407] Connection header received
0807 18:33:08.803251 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:08.803253 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:08.803316 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:08.803392 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:33:08.804195 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:08.804773 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:08.805932 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:08.806134 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:08.806213 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:08.807505 (+ 1292us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:33:08.807508 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:33:08.807509 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:33:08.807531 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:33:08.809016 (+ 1485us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:08.809526 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:08.809529 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:08.809530 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:08.809604 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:08.809850 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:08.809852 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:08.809853 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:08.810004 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:33:08.810080 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:08.810264 (+ 184us) server_negotiation.cc:299] Negotiation successful
0807 18:33:08.810349 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260807 18:33:08.810505 12776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:08.802601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:34397 (local address 127.11.14.65:35579)
0807 18:33:08.803039 (+ 438us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:08.803056 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:08.803165 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:08.803412 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:08.803415 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:08.803811 (+ 396us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:33:08.804067 (+ 256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:08.804075 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:08.804938 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:08.804941 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:08.805821 (+ 880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:08.805831 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:08.806253 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:08.806257 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:08.806361 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:08.806974 (+ 613us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:33:08.806988 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:33:08.807375 (+ 387us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:33:08.809195 (+ 1820us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:33:08.809200 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:33:08.809202 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:33:08.809435 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:33:08.809687 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:33:08.809689 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:33:08.809690 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:33:08.809765 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:33:08.810083 (+ 318us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:33:08.810085 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:33:08.810154 (+ 69us) client_negotiation.cc:769] Sending connection context
0807 18:33:08.810244 (+ 90us) client_negotiation.cc:240] Negotiation successful
0807 18:33:08.810342 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":303,"thread_start_us":131,"threads_started":1}
I20260807 18:33:08.811244 11321 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkz56XN/build/debug/bin/kudu as pid 12621
I20260807 18:33:08.811388 11321 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127579900523-11321-0/minicluster-data/ts-0/wal/instance
I20260807 18:33:08.811731 11321 external_mini_cluster.cc:1428] Setting key f00f06ad807c5c2c1e8b52a97774b431
WARNING: no policy specified for kudu/127.11.14.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.14.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.14.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.66.keytab.
Entry for principal kudu/127.11.14.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.66.keytab.
I20260807 18:33:08.842459 11321 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkz56XN/build/debug/bin/kudu
/tmp/dist-test-taskkz56XN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127579900523-11321-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127579900523-11321-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127579900523-11321-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127579900523-11321-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.14.66:0
--local_ip_for_outbound_sockets=127.11.14.66
--webserver_interface=127.11.14.66
--webserver_port=0
--tserver_master_addrs=127.11.14.126:34397
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.66.keytab
--principal=kudu/127.11.14.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.14.84:46775
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127579900523-11321-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127579900523-11321-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127579900523-11321-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:33:08.983740 12779 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:33:08.984057 12779 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:33:08.984138 12779 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:33:08.988459 12779 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:33:08.988567 12779 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:33:08.988644 12779 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 18:33:08.988737 12779 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.14.66
I20260807 18:33:08.994290 12779 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.14.84:46775
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127579900523-11321-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127579900523-11321-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.66.keytab
--principal=kudu/127.11.14.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.14.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127579900523-11321-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.14.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.11.14.126:34397
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12779
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.14.66
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127579900523-11321-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:33:08.995397 12779 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:33:08.996492 12779 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:33:09.007203 12787 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:33:09.007313 12785 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:33:09.007324 12784 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:33:09.007467 12779 server_base.cc:1034] running on GCE node
I20260807 18:33:09.007984 12779 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:33:09.008643 12779 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:33:09.009876 12779 hybrid_clock.cc:648] HybridClock initialized: now 1786127589009859 us; error 43 us; skew 500 ppm
Aug 07 18:33:09 dist-test-slave-6k22 krb5kdc[12492](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127589, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:33:09.013254 12779 init.cc:377] Logged in from keytab as kudu/127.11.14.66@KRBTEST.COM (short username kudu)
I20260807 18:33:09.014199 12779 webserver.cc:460] Webserver started at http://127.11.14.66:42967/ using document root <none> and password file <none>
I20260807 18:33:09.014794 12779 fs_manager.cc:359] Metadata directory not provided
I20260807 18:33:09.014889 12779 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:33:09.015120 12779 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:33:09.017388 12779 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127579900523-11321-0/minicluster-data/ts-1/data/instance:
uuid: "34ede5db17834cfe86abe88e470bc41c"
format_stamp: "Formatted at 2026-08-07 18:33:09 on dist-test-slave-6k22"
server_key: "e5ca2c659bfd8b81a0d27f5314704811"
server_key_iv: "571f1f7eded5bd8b04a3ff5b6d3ff524"
server_key_version: "encryptionkey@0"
I20260807 18:33:09.018026 12779 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127579900523-11321-0/minicluster-data/ts-1/wal/instance:
uuid: "34ede5db17834cfe86abe88e470bc41c"
format_stamp: "Formatted at 2026-08-07 18:33:09 on dist-test-slave-6k22"
server_key: "e5ca2c659bfd8b81a0d27f5314704811"
server_key_iv: "571f1f7eded5bd8b04a3ff5b6d3ff524"
server_key_version: "encryptionkey@0"
I20260807 18:33:09.022023 12779 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.001s
I20260807 18:33:09.024663 12794 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:33:09.025873 12779 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 18:33:09.026011 12779 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127579900523-11321-0/minicluster-data/ts-1/data,/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127579900523-11321-0/minicluster-data/ts-1/wal
uuid: "34ede5db17834cfe86abe88e470bc41c"
format_stamp: "Formatted at 2026-08-07 18:33:09 on dist-test-slave-6k22"
server_key: "e5ca2c659bfd8b81a0d27f5314704811"
server_key_iv: "571f1f7eded5bd8b04a3ff5b6d3ff524"
server_key_version: "encryptionkey@0"
I20260807 18:33:09.026126 12779 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127579900523-11321-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127579900523-11321-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127579900523-11321-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:33:09.037756 12779 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:33:09.041293 12779 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:33:09.041529 12779 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:33:09.043190 12779 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:33:09.043260 12779 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:33:09.043332 12779 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:33:09.043376 12779 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 18:33:09 dist-test-slave-6k22 krb5kdc[12492](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127589, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.66@KRBTEST.COM for kudu/127.11.14.126@KRBTEST.COM
I20260807 18:33:09.059449 12779 rpc_server.cc:307] RPC server started. Bound to: 127.11.14.66:44069
I20260807 18:33:09.059783 12922 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.14.66:44069 every 8 connection(s)
I20260807 18:33:09.060272 12597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:09.044124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.66:59861 (local address 127.11.14.126:34397)
0807 18:33:09.044330 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:09.044338 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:09.045850 (+ 1512us) server_negotiation.cc:407] Connection header received
0807 18:33:09.046977 (+ 1127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:09.046981 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:09.047045 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:09.047137 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:33:09.048846 (+ 1709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:09.049680 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:09.050934 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:09.051257 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:09.051372 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:09.055759 (+ 4387us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:33:09.055766 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:33:09.055770 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:33:09.055803 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:33:09.057768 (+ 1965us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:09.058510 (+ 742us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:09.058513 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:09.058514 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:09.058594 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:09.059016 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:09.059022 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:09.059027 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:09.059300 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:33:09.059416 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:09.060040 (+ 624us) server_negotiation.cc:299] Negotiation successful
0807 18:33:09.060142 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260807 18:33:09.060560 12779 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127579900523-11321-0/minicluster-data/ts-1/data/info.pb
I20260807 18:33:09.061776 12804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:09.044981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:34397 (local address 127.11.14.66:59861)
0807 18:33:09.045610 (+ 629us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:09.045663 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:09.046712 (+ 1049us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:09.047315 (+ 603us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:09.047323 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:09.047894 (+ 571us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:33:09.048676 (+ 782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:09.048700 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:09.049806 (+ 1106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:09.049810 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:09.050778 (+ 968us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:09.050788 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:09.051438 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:09.051442 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:09.051705 (+ 263us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:09.052535 (+ 830us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:33:09.052554 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:33:09.055446 (+ 2892us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:33:09.057933 (+ 2487us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:33:09.057941 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:33:09.057957 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:33:09.058401 (+ 444us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:33:09.058689 (+ 288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:33:09.058692 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:33:09.058694 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:33:09.058887 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:33:09.059427 (+ 540us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:33:09.059433 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:33:09.059769 (+ 336us) client_negotiation.cc:769] Sending connection context
0807 18:33:09.059974 (+ 205us) client_negotiation.cc:240] Negotiation successful
0807 18:33:09.060324 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":376,"thread_start_us":254,"threads_started":1}
I20260807 18:33:09.062952 12923 heartbeater.cc:347] Connected to a master server at 127.11.14.126:34397
I20260807 18:33:09.063113 12923 heartbeater.cc:464] Registering TS with master...
I20260807 18:33:09.063686 12923 heartbeater.cc:511] Master 127.11.14.126:34397 requested a full tablet report, sending...
I20260807 18:33:09.064812 12542 ts_manager.cc:194] Registered new tserver with Master: 34ede5db17834cfe86abe88e470bc41c (127.11.14.66:44069)
I20260807 18:33:09.065574 12542 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.14.66@KRBTEST.COM'} at 127.11.14.66:59861
I20260807 18:33:09.069576 11321 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkz56XN/build/debug/bin/kudu as pid 12779
I20260807 18:33:09.069741 11321 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127579900523-11321-0/minicluster-data/ts-1/wal/instance
I20260807 18:33:09.070060 11321 external_mini_cluster.cc:1428] Setting key cfe0064fb1d7a1ab8af855793e5a623b
I20260807 18:33:09.073222 12597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:09.065514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.66:47259 (local address 127.11.14.126:34397)
0807 18:33:09.065642 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:09.065646 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:09.066024 (+ 378us) server_negotiation.cc:407] Connection header received
0807 18:33:09.066140 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:09.066144 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:09.066191 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:09.066281 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:33:09.066881 (+ 600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:09.067387 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:09.068314 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:09.068539 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:09.068650 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:09.069860 (+ 1210us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:33:09.069864 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:33:09.069867 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:33:09.069895 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:33:09.071504 (+ 1609us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:09.072035 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:09.072037 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:09.072038 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:09.072110 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:09.072411 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:09.072414 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:09.072415 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:09.072603 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:33:09.072679 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:09.073006 (+ 327us) server_negotiation.cc:299] Negotiation successful
0807 18:33:09.073110 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260807 18:33:09.073419 12929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:09.065409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:34397 (local address 127.11.14.66:47259)
0807 18:33:09.065937 (+ 528us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:09.065949 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:09.066065 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:09.066297 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:09.066300 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:09.066585 (+ 285us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:33:09.066762 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:09.066770 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:09.067521 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:09.067524 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:09.068191 (+ 667us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:09.068201 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:09.068653 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:09.068656 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:09.068758 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:09.069255 (+ 497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:33:09.069275 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:33:09.069656 (+ 381us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:33:09.071671 (+ 2015us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:33:09.071674 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:33:09.071676 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:33:09.071936 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:33:09.072203 (+ 267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:33:09.072208 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:33:09.072213 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:33:09.072318 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:33:09.072698 (+ 380us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:33:09.072704 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:33:09.072825 (+ 121us) client_negotiation.cc:769] Sending connection context
0807 18:33:09.072955 (+ 130us) client_negotiation.cc:240] Negotiation successful
0807 18:33:09.073138 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":458,"mutex_wait_us":2,"thread_start_us":150,"threads_started":1}
WARNING: no policy specified for kudu/127.11.14.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.14.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.14.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.67.keytab.
Entry for principal kudu/127.11.14.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.67.keytab.
I20260807 18:33:09.102691 11321 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkz56XN/build/debug/bin/kudu
/tmp/dist-test-taskkz56XN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127579900523-11321-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127579900523-11321-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127579900523-11321-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127579900523-11321-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.14.67:0
--local_ip_for_outbound_sockets=127.11.14.67
--webserver_interface=127.11.14.67
--webserver_port=0
--tserver_master_addrs=127.11.14.126:34397
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.67.keytab
--principal=kudu/127.11.14.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.14.84:46775
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127579900523-11321-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127579900523-11321-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127579900523-11321-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:33:09.226616 12932 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:33:09.226903 12932 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:33:09.226990 12932 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:33:09.231122 12932 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:33:09.231199 12932 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:33:09.231266 12932 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 18:33:09.231343 12932 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.14.67
I20260807 18:33:09.236471 12932 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.14.84:46775
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127579900523-11321-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127579900523-11321-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.67.keytab
--principal=kudu/127.11.14.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.14.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127579900523-11321-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.14.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.11.14.126:34397
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12932
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.14.67
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127579900523-11321-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:33:09.237517 12932 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:33:09.238651 12932 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:33:09.248669 12937 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:33:09.248679 12940 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:33:09.249030 12932 server_base.cc:1034] running on GCE node
W20260807 18:33:09.248680 12938 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:33:09.249547 12932 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:33:09.250162 12932 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:33:09.251330 12932 hybrid_clock.cc:648] HybridClock initialized: now 1786127589251312 us; error 29 us; skew 500 ppm
Aug 07 18:33:09 dist-test-slave-6k22 krb5kdc[12492](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127589, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:33:09.254366 12932 init.cc:377] Logged in from keytab as kudu/127.11.14.67@KRBTEST.COM (short username kudu)
I20260807 18:33:09.255272 12932 webserver.cc:460] Webserver started at http://127.11.14.67:45647/ using document root <none> and password file <none>
I20260807 18:33:09.255924 12932 fs_manager.cc:359] Metadata directory not provided
I20260807 18:33:09.256021 12932 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:33:09.256260 12932 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:33:09.258308 12932 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127579900523-11321-0/minicluster-data/ts-2/data/instance:
uuid: "a4ff3361f5784240bf29f7016bdaaa42"
format_stamp: "Formatted at 2026-08-07 18:33:09 on dist-test-slave-6k22"
server_key: "0d2fd9295c8d8f1b4e23b5bc42775919"
server_key_iv: "0d38476e109897287ecf02868e241072"
server_key_version: "encryptionkey@0"
I20260807 18:33:09.258911 12932 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127579900523-11321-0/minicluster-data/ts-2/wal/instance:
uuid: "a4ff3361f5784240bf29f7016bdaaa42"
format_stamp: "Formatted at 2026-08-07 18:33:09 on dist-test-slave-6k22"
server_key: "0d2fd9295c8d8f1b4e23b5bc42775919"
server_key_iv: "0d38476e109897287ecf02868e241072"
server_key_version: "encryptionkey@0"
I20260807 18:33:09.262939 12932 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 18:33:09.265516 12948 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:33:09.266733 12932 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 18:33:09.266881 12932 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127579900523-11321-0/minicluster-data/ts-2/data,/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127579900523-11321-0/minicluster-data/ts-2/wal
uuid: "a4ff3361f5784240bf29f7016bdaaa42"
format_stamp: "Formatted at 2026-08-07 18:33:09 on dist-test-slave-6k22"
server_key: "0d2fd9295c8d8f1b4e23b5bc42775919"
server_key_iv: "0d38476e109897287ecf02868e241072"
server_key_version: "encryptionkey@0"
I20260807 18:33:09.266995 12932 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127579900523-11321-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127579900523-11321-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127579900523-11321-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:33:09.280767 12932 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:33:09.283808 12932 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:33:09.284015 12932 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:33:09.285584 12932 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:33:09.285660 12932 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:33:09.285733 12932 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:33:09.285792 12932 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 18:33:09 dist-test-slave-6k22 krb5kdc[12492](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127589, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.67@KRBTEST.COM for kudu/127.11.14.126@KRBTEST.COM
I20260807 18:33:09.297871 12932 rpc_server.cc:307] RPC server started. Bound to: 127.11.14.67:46029
I20260807 18:33:09.297917 13075 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.14.67:46029 every 8 connection(s)
I20260807 18:33:09.298938 12932 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127579900523-11321-0/minicluster-data/ts-2/data/info.pb
I20260807 18:33:09.300068 11321 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkz56XN/build/debug/bin/kudu as pid 12932
I20260807 18:33:09.300223 11321 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127579900523-11321-0/minicluster-data/ts-2/wal/instance
I20260807 18:33:09.300499 11321 external_mini_cluster.cc:1428] Setting key 2705f30376a7a53164099f96685d7333
I20260807 18:33:09.300984 12597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:09.286354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.67:35883 (local address 127.11.14.126:34397)
0807 18:33:09.286487 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:09.286491 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:09.288242 (+ 1751us) server_negotiation.cc:407] Connection header received
0807 18:33:09.289434 (+ 1192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:09.289437 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:09.289499 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:09.289606 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:33:09.290909 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:09.291538 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:09.292907 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:09.293208 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:09.293331 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:09.296445 (+ 3114us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:33:09.296449 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:33:09.296452 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:33:09.296485 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:33:09.298112 (+ 1627us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:09.298745 (+ 633us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:09.298751 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:09.298755 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:09.298892 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:09.299296 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:09.299319 (+ 23us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:09.299321 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:09.299732 (+ 411us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:33:09.299859 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:09.300641 (+ 782us) server_negotiation.cc:299] Negotiation successful
0807 18:33:09.300788 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260807 18:33:09.301962 12958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:09.287492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:34397 (local address 127.11.14.67:35883)
0807 18:33:09.288080 (+ 588us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:09.288128 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:09.289217 (+ 1089us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:09.289800 (+ 583us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:09.289808 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:09.290230 (+ 422us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:33:09.290761 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:09.290774 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:09.291725 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:09.291736 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:09.292766 (+ 1030us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:09.292778 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:09.293384 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:09.293389 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:09.293614 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:09.294314 (+ 700us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:33:09.294334 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:33:09.296267 (+ 1933us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:33:09.298230 (+ 1963us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:33:09.298238 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:33:09.298254 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:33:09.298605 (+ 351us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:33:09.298985 (+ 380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:33:09.298991 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:33:09.298995 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:33:09.299178 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:33:09.299956 (+ 778us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:33:09.299963 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:33:09.300332 (+ 369us) client_negotiation.cc:769] Sending connection context
0807 18:33:09.300573 (+ 241us) client_negotiation.cc:240] Negotiation successful
0807 18:33:09.301057 (+ 484us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":342,"thread_start_us":143,"threads_started":1}
I20260807 18:33:09.303336 13076 heartbeater.cc:347] Connected to a master server at 127.11.14.126:34397
I20260807 18:33:09.303499 13076 heartbeater.cc:464] Registering TS with master...
I20260807 18:33:09.304068 13076 heartbeater.cc:511] Master 127.11.14.126:34397 requested a full tablet report, sending...
I20260807 18:33:09.305121 12543 ts_manager.cc:194] Registered new tserver with Master: a4ff3361f5784240bf29f7016bdaaa42 (127.11.14.67:46029)
I20260807 18:33:09.305896 12543 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.14.67@KRBTEST.COM'} at 127.11.14.67:35883
I20260807 18:33:09.312740 12597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:09.305631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.67:35365 (local address 127.11.14.126:34397)
0807 18:33:09.305764 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:09.305768 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:09.305947 (+ 179us) server_negotiation.cc:407] Connection header received
0807 18:33:09.306112 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:09.306115 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:09.306182 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:09.306266 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:33:09.306795 (+ 529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:09.307528 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:09.308351 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:09.308557 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:09.308638 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:09.309673 (+ 1035us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:33:09.309676 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:33:09.309678 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:33:09.309700 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:33:09.311069 (+ 1369us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:09.311457 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:09.311459 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:09.311461 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:09.311533 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:09.311900 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:09.311903 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:09.311905 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:09.312082 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:33:09.312194 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:09.312462 (+ 268us) server_negotiation.cc:299] Negotiation successful
0807 18:33:09.312573 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260807 18:33:09.312762 13082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:09.305515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:34397 (local address 127.11.14.67:35365)
0807 18:33:09.305857 (+ 342us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:09.305872 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:09.306021 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:09.306299 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:09.306302 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:09.306530 (+ 228us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:33:09.306665 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:09.306679 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:09.307672 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:09.307675 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:09.308223 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:09.308231 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:09.308658 (+ 427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:09.308660 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:09.308728 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:09.309201 (+ 473us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:33:09.309212 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:33:09.309549 (+ 337us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:33:09.311171 (+ 1622us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:33:09.311174 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:33:09.311175 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:33:09.311372 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:33:09.311692 (+ 320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:33:09.311695 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:33:09.311698 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:33:09.311790 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:33:09.312185 (+ 395us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:33:09.312188 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:33:09.312289 (+ 101us) client_negotiation.cc:769] Sending connection context
0807 18:33:09.312402 (+ 113us) client_negotiation.cc:240] Negotiation successful
0807 18:33:09.312555 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":256,"mutex_wait_us":2,"thread_start_us":146,"threads_started":1}
I20260807 18:33:09.314672 11321 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 18:33:09 dist-test-slave-6k22 krb5kdc[12492](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127589, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 07 18:33:09 dist-test-slave-6k22 krb5kdc[12492](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127589, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.11.14.126@KRBTEST.COM
I20260807 18:33:09.342411 12597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:09.334375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53650 (local address 127.11.14.126:34397)
0807 18:33:09.334516 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:09.334520 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:09.334532 (+ 12us) server_negotiation.cc:407] Connection header received
0807 18:33:09.334596 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:09.334599 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:09.334638 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:09.334708 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:33:09.335456 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:09.336060 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:09.336892 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:09.337098 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:09.337178 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:09.339629 (+ 2451us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:33:09.339632 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:33:09.339634 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:33:09.339659 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:33:09.340940 (+ 1281us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:09.341316 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:09.341319 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:09.341320 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:09.341394 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:09.341624 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:09.341626 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:09.341627 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:09.341798 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:33:09.341910 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:09.342113 (+ 203us) server_negotiation.cc:299] Negotiation successful
0807 18:33:09.342210 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260807 18:33:09.345613 12542 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53650:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 18:33:09.348816 12542 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 18:33:09.363679 13096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:09.357622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57286 (local address 127.11.14.67:46029)
0807 18:33:09.357962 (+ 340us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:09.357975 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:09.358297 (+ 322us) server_negotiation.cc:407] Connection header received
0807 18:33:09.358534 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:09.358541 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:09.358782 (+ 241us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:09.358947 (+ 165us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:33:09.359295 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:09.360470 (+ 1175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:09.362148 (+ 1678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:09.362934 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:09.363059 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:09.363246 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:09.363362 (+ 116us) server_negotiation.cc:299] Negotiation successful
0807 18:33:09.363458 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":233,"thread_start_us":104,"threads_started":1}
I20260807 18:33:09.363672 13093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:09.357246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.67:46029 (local address 127.0.0.1:57286)
0807 18:33:09.358129 (+ 883us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:09.358182 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:09.358354 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:09.359006 (+ 652us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:09.359012 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:09.359035 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:33:09.359170 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:09.359176 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:09.360591 (+ 1415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:09.360594 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:09.362018 (+ 1424us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:09.362029 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:09.363059 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:09.363063 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:09.363169 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:09.363208 (+ 39us) client_negotiation.cc:769] Sending connection context
0807 18:33:09.363336 (+ 128us) client_negotiation.cc:240] Negotiation successful
0807 18:33:09.363468 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":663,"mutex_wait_us":32,"thread_start_us":93,"threads_started":1}
I20260807 18:33:09.364015 13095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:09.357549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53726 (local address 127.11.14.65:44813)
0807 18:33:09.358177 (+ 628us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:09.358184 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:09.358998 (+ 814us) server_negotiation.cc:407] Connection header received
0807 18:33:09.359088 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:09.359093 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:09.359258 (+ 165us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:09.359412 (+ 154us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:33:09.359695 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:09.360741 (+ 1046us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:09.362247 (+ 1506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:09.363057 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:09.363157 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:09.363302 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:09.363754 (+ 452us) server_negotiation.cc:299] Negotiation successful
0807 18:33:09.363878 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":528,"thread_start_us":64,"threads_started":1}
I20260807 18:33:09.364562 13097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:09.357357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.65:44813 (local address 127.0.0.1:53726)
0807 18:33:09.358763 (+ 1406us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:09.358775 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:09.358890 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:09.359401 (+ 511us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:09.359405 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:09.359424 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:33:09.359520 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:09.359528 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:09.360891 (+ 1363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:09.360894 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:09.362103 (+ 1209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:09.362117 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:09.363374 (+ 1257us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:09.363378 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:09.363474 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:09.363497 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 18:33:09.363594 (+ 97us) client_negotiation.cc:240] Negotiation successful
0807 18:33:09.363667 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1339,"mutex_wait_us":60,"thread_start_us":93,"threads_started":1}
I20260807 18:33:09.364796 13094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:09.358042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.66:44069 (local address 127.0.0.1:54876)
0807 18:33:09.359677 (+ 1635us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:09.359689 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:09.359806 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:09.360204 (+ 398us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:09.360207 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:09.360225 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:33:09.360310 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:09.360317 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:09.361793 (+ 1476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:09.361796 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:09.362988 (+ 1192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:09.363001 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:09.364415 (+ 1414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:09.364419 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:09.364504 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:09.364527 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 18:33:09.364603 (+ 76us) client_negotiation.cc:240] Negotiation successful
0807 18:33:09.364666 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1560,"mutex_wait_us":17,"thread_start_us":95,"threads_started":1}
I20260807 18:33:09.365259 13098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:09.358660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54876 (local address 127.11.14.66:44069)
0807 18:33:09.359016 (+ 356us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:09.359037 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:09.359793 (+ 756us) server_negotiation.cc:407] Connection header received
0807 18:33:09.359931 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:09.359935 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:09.360074 (+ 139us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:09.360222 (+ 148us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:33:09.360407 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:09.361665 (+ 1258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:09.363146 (+ 1481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:09.364002 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:09.364118 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:09.364314 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:09.365022 (+ 708us) server_negotiation.cc:299] Negotiation successful
0807 18:33:09.365121 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":252,"thread_start_us":123,"threads_started":1}
I20260807 18:33:09.367486 12849 tablet_service.cc:1467] Processing CreateTablet for tablet fe70cc275fab45d591f7bff25fbf59dd (DEFAULT_TABLE table=test-table [id=9be290545f254a99a1ea5a42abbf3c10]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:33:09.367494 13004 tablet_service.cc:1467] Processing CreateTablet for tablet fe70cc275fab45d591f7bff25fbf59dd (DEFAULT_TABLE table=test-table [id=9be290545f254a99a1ea5a42abbf3c10]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:33:09.367489 12694 tablet_service.cc:1467] Processing CreateTablet for tablet fe70cc275fab45d591f7bff25fbf59dd (DEFAULT_TABLE table=test-table [id=9be290545f254a99a1ea5a42abbf3c10]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:33:09.368628 12849 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fe70cc275fab45d591f7bff25fbf59dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:33:09.368647 12694 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fe70cc275fab45d591f7bff25fbf59dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:33:09.368659 13004 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fe70cc275fab45d591f7bff25fbf59dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:33:09.376494 13099 tablet_bootstrap.cc:492] T fe70cc275fab45d591f7bff25fbf59dd P 34ede5db17834cfe86abe88e470bc41c: Bootstrap starting.
I20260807 18:33:09.377629 13101 tablet_bootstrap.cc:492] T fe70cc275fab45d591f7bff25fbf59dd P a4ff3361f5784240bf29f7016bdaaa42: Bootstrap starting.
I20260807 18:33:09.377861 13100 tablet_bootstrap.cc:492] T fe70cc275fab45d591f7bff25fbf59dd P 62c0ffbc8d8d46fdb61ff8f094e1bb68: Bootstrap starting.
I20260807 18:33:09.378851 13099 tablet_bootstrap.cc:654] T fe70cc275fab45d591f7bff25fbf59dd P 34ede5db17834cfe86abe88e470bc41c: Neither blocks nor log segments found. Creating new log.
I20260807 18:33:09.379700 13099 log.cc:824] T fe70cc275fab45d591f7bff25fbf59dd P 34ede5db17834cfe86abe88e470bc41c: Log is configured to *not* fsync() on all Append() calls
I20260807 18:33:09.379824 13101 tablet_bootstrap.cc:654] T fe70cc275fab45d591f7bff25fbf59dd P a4ff3361f5784240bf29f7016bdaaa42: Neither blocks nor log segments found. Creating new log.
I20260807 18:33:09.379824 13100 tablet_bootstrap.cc:654] T fe70cc275fab45d591f7bff25fbf59dd P 62c0ffbc8d8d46fdb61ff8f094e1bb68: Neither blocks nor log segments found. Creating new log.
I20260807 18:33:09.380666 13100 log.cc:824] T fe70cc275fab45d591f7bff25fbf59dd P 62c0ffbc8d8d46fdb61ff8f094e1bb68: Log is configured to *not* fsync() on all Append() calls
I20260807 18:33:09.380666 13101 log.cc:824] T fe70cc275fab45d591f7bff25fbf59dd P a4ff3361f5784240bf29f7016bdaaa42: Log is configured to *not* fsync() on all Append() calls
I20260807 18:33:09.381827 13099 tablet_bootstrap.cc:492] T fe70cc275fab45d591f7bff25fbf59dd P 34ede5db17834cfe86abe88e470bc41c: No bootstrap required, opened a new log
I20260807 18:33:09.382066 13099 ts_tablet_manager.cc:1397] T fe70cc275fab45d591f7bff25fbf59dd P 34ede5db17834cfe86abe88e470bc41c: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260807 18:33:09.382581 13101 tablet_bootstrap.cc:492] T fe70cc275fab45d591f7bff25fbf59dd P a4ff3361f5784240bf29f7016bdaaa42: No bootstrap required, opened a new log
I20260807 18:33:09.382782 13101 ts_tablet_manager.cc:1397] T fe70cc275fab45d591f7bff25fbf59dd P a4ff3361f5784240bf29f7016bdaaa42: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260807 18:33:09.384925 13100 tablet_bootstrap.cc:492] T fe70cc275fab45d591f7bff25fbf59dd P 62c0ffbc8d8d46fdb61ff8f094e1bb68: No bootstrap required, opened a new log
I20260807 18:33:09.385115 13100 ts_tablet_manager.cc:1397] T fe70cc275fab45d591f7bff25fbf59dd P 62c0ffbc8d8d46fdb61ff8f094e1bb68: Time spent bootstrapping tablet: real 0.007s user 0.003s sys 0.000s
I20260807 18:33:09.385329 13099 raft_consensus.cc:348] T fe70cc275fab45d591f7bff25fbf59dd P 34ede5db17834cfe86abe88e470bc41c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4ff3361f5784240bf29f7016bdaaa42" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 46029 } } peers { permanent_uuid: "62c0ffbc8d8d46fdb61ff8f094e1bb68" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 44813 } } peers { permanent_uuid: "34ede5db17834cfe86abe88e470bc41c" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 44069 } }
I20260807 18:33:09.385577 13099 raft_consensus.cc:374] T fe70cc275fab45d591f7bff25fbf59dd P 34ede5db17834cfe86abe88e470bc41c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:33:09.385634 13099 raft_consensus.cc:729] T fe70cc275fab45d591f7bff25fbf59dd P 34ede5db17834cfe86abe88e470bc41c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34ede5db17834cfe86abe88e470bc41c, State: Initialized, Role: FOLLOWER
I20260807 18:33:09.386093 13099 consensus_queue.cc:260] T fe70cc275fab45d591f7bff25fbf59dd P 34ede5db17834cfe86abe88e470bc41c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4ff3361f5784240bf29f7016bdaaa42" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 46029 } } peers { permanent_uuid: "62c0ffbc8d8d46fdb61ff8f094e1bb68" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 44813 } } peers { permanent_uuid: "34ede5db17834cfe86abe88e470bc41c" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 44069 } }
I20260807 18:33:09.386864 12923 heartbeater.cc:503] Master 127.11.14.126:34397 was elected leader, sending a full tablet report...
I20260807 18:33:09.386693 13101 raft_consensus.cc:348] T fe70cc275fab45d591f7bff25fbf59dd P a4ff3361f5784240bf29f7016bdaaa42 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4ff3361f5784240bf29f7016bdaaa42" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 46029 } } peers { permanent_uuid: "62c0ffbc8d8d46fdb61ff8f094e1bb68" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 44813 } } peers { permanent_uuid: "34ede5db17834cfe86abe88e470bc41c" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 44069 } }
I20260807 18:33:09.386950 13101 raft_consensus.cc:374] T fe70cc275fab45d591f7bff25fbf59dd P a4ff3361f5784240bf29f7016bdaaa42 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:33:09.386998 13101 raft_consensus.cc:729] T fe70cc275fab45d591f7bff25fbf59dd P a4ff3361f5784240bf29f7016bdaaa42 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4ff3361f5784240bf29f7016bdaaa42, State: Initialized, Role: FOLLOWER
I20260807 18:33:09.387274 13099 ts_tablet_manager.cc:1428] T fe70cc275fab45d591f7bff25fbf59dd P 34ede5db17834cfe86abe88e470bc41c: Time spent starting tablet: real 0.005s user 0.007s sys 0.000s
I20260807 18:33:09.387416 13101 consensus_queue.cc:260] T fe70cc275fab45d591f7bff25fbf59dd P a4ff3361f5784240bf29f7016bdaaa42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4ff3361f5784240bf29f7016bdaaa42" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 46029 } } peers { permanent_uuid: "62c0ffbc8d8d46fdb61ff8f094e1bb68" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 44813 } } peers { permanent_uuid: "34ede5db17834cfe86abe88e470bc41c" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 44069 } }
I20260807 18:33:09.388155 13076 heartbeater.cc:503] Master 127.11.14.126:34397 was elected leader, sending a full tablet report...
I20260807 18:33:09.388427 13101 ts_tablet_manager.cc:1428] T fe70cc275fab45d591f7bff25fbf59dd P a4ff3361f5784240bf29f7016bdaaa42: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260807 18:33:09.389617 13100 raft_consensus.cc:348] T fe70cc275fab45d591f7bff25fbf59dd P 62c0ffbc8d8d46fdb61ff8f094e1bb68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4ff3361f5784240bf29f7016bdaaa42" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 46029 } } peers { permanent_uuid: "62c0ffbc8d8d46fdb61ff8f094e1bb68" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 44813 } } peers { permanent_uuid: "34ede5db17834cfe86abe88e470bc41c" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 44069 } }
I20260807 18:33:09.389920 13100 raft_consensus.cc:374] T fe70cc275fab45d591f7bff25fbf59dd P 62c0ffbc8d8d46fdb61ff8f094e1bb68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:33:09.389989 13100 raft_consensus.cc:729] T fe70cc275fab45d591f7bff25fbf59dd P 62c0ffbc8d8d46fdb61ff8f094e1bb68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 62c0ffbc8d8d46fdb61ff8f094e1bb68, State: Initialized, Role: FOLLOWER
I20260807 18:33:09.390470 13100 consensus_queue.cc:260] T fe70cc275fab45d591f7bff25fbf59dd P 62c0ffbc8d8d46fdb61ff8f094e1bb68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4ff3361f5784240bf29f7016bdaaa42" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 46029 } } peers { permanent_uuid: "62c0ffbc8d8d46fdb61ff8f094e1bb68" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 44813 } } peers { permanent_uuid: "34ede5db17834cfe86abe88e470bc41c" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 44069 } }
I20260807 18:33:09.391245 12773 heartbeater.cc:503] Master 127.11.14.126:34397 was elected leader, sending a full tablet report...
I20260807 18:33:09.392580 13100 ts_tablet_manager.cc:1428] T fe70cc275fab45d591f7bff25fbf59dd P 62c0ffbc8d8d46fdb61ff8f094e1bb68: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260807 18:33:09.457723 13108 raft_consensus.cc:482] T fe70cc275fab45d591f7bff25fbf59dd P a4ff3361f5784240bf29f7016bdaaa42 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:33:09.457953 13108 raft_consensus.cc:504] T fe70cc275fab45d591f7bff25fbf59dd P a4ff3361f5784240bf29f7016bdaaa42 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4ff3361f5784240bf29f7016bdaaa42" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 46029 } } peers { permanent_uuid: "62c0ffbc8d8d46fdb61ff8f094e1bb68" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 44813 } } peers { permanent_uuid: "34ede5db17834cfe86abe88e470bc41c" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 44069 } }
I20260807 18:33:09.459120 13108 leader_election.cc:288] T fe70cc275fab45d591f7bff25fbf59dd P a4ff3361f5784240bf29f7016bdaaa42 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 62c0ffbc8d8d46fdb61ff8f094e1bb68 (127.11.14.65:44813), 34ede5db17834cfe86abe88e470bc41c (127.11.14.66:44069)
I20260807 18:33:09.464076 13098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:09.459946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.67:35223 (local address 127.11.14.66:44069)
0807 18:33:09.460087 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:09.460092 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:09.460361 (+ 269us) server_negotiation.cc:407] Connection header received
0807 18:33:09.460448 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:09.460450 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:09.460491 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:09.460550 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:33:09.460850 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:09.461565 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:09.462944 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:09.463414 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:09.463548 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:09.463650 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:09.463855 (+ 205us) server_negotiation.cc:299] Negotiation successful
0807 18:33:09.463940 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260807 18:33:09.464094 13110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:09.459776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.66:44069 (local address 127.11.14.67:35223)
0807 18:33:09.460249 (+ 473us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:09.460263 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:09.460367 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:09.460587 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:09.460590 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:09.460608 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:33:09.460743 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:09.460748 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:09.461783 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:09.461788 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:09.462793 (+ 1005us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:09.462803 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:09.463544 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:09.463547 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:09.463710 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:09.463756 (+ 46us) client_negotiation.cc:769] Sending connection context
0807 18:33:09.463835 (+ 79us) client_negotiation.cc:240] Negotiation successful
0807 18:33:09.463904 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":393,"mutex_wait_us":39,"thread_start_us":161,"threads_started":1}
I20260807 18:33:09.464110 13095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:09.459549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.67:57361 (local address 127.11.14.65:44813)
0807 18:33:09.459702 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:09.459708 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:09.459723 (+ 15us) server_negotiation.cc:407] Connection header received
0807 18:33:09.459803 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:09.459806 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:09.459862 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:09.459943 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:33:09.460294 (+ 351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:09.461026 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:09.462964 (+ 1938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:09.463414 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:09.463548 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:09.463640 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:09.463898 (+ 258us) server_negotiation.cc:299] Negotiation successful
0807 18:33:09.463982 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260807 18:33:09.464174 12958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:09.459424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.65:44813 (local address 127.11.14.67:57361)
0807 18:33:09.459589 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:09.459606 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:09.459708 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:09.459985 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:09.459988 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:09.460012 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:33:09.460161 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:09.460167 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:09.461142 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:09.461146 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:09.462815 (+ 1669us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:09.462827 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:09.463545 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:09.463549 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:09.463727 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:09.463757 (+ 30us) client_negotiation.cc:769] Sending connection context
0807 18:33:09.463874 (+ 117us) client_negotiation.cc:240] Negotiation successful
0807 18:33:09.463955 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50,"mutex_wait_us":31}
I20260807 18:33:09.464723 12872 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe70cc275fab45d591f7bff25fbf59dd" candidate_uuid: "a4ff3361f5784240bf29f7016bdaaa42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34ede5db17834cfe86abe88e470bc41c" is_pre_election: true
I20260807 18:33:09.464726 12716 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe70cc275fab45d591f7bff25fbf59dd" candidate_uuid: "a4ff3361f5784240bf29f7016bdaaa42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62c0ffbc8d8d46fdb61ff8f094e1bb68" is_pre_election: true
I20260807 18:33:09.465057 12716 raft_consensus.cc:2454] T fe70cc275fab45d591f7bff25fbf59dd P 62c0ffbc8d8d46fdb61ff8f094e1bb68 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4ff3361f5784240bf29f7016bdaaa42 in term 0.
I20260807 18:33:09.465057 12872 raft_consensus.cc:2454] T fe70cc275fab45d591f7bff25fbf59dd P 34ede5db17834cfe86abe88e470bc41c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4ff3361f5784240bf29f7016bdaaa42 in term 0.
I20260807 18:33:09.465552 12952 leader_election.cc:302] T fe70cc275fab45d591f7bff25fbf59dd P a4ff3361f5784240bf29f7016bdaaa42 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 62c0ffbc8d8d46fdb61ff8f094e1bb68, a4ff3361f5784240bf29f7016bdaaa42; no voters:
I20260807 18:33:09.465791 13108 raft_consensus.cc:2781] T fe70cc275fab45d591f7bff25fbf59dd P a4ff3361f5784240bf29f7016bdaaa42 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:33:09.465878 13108 raft_consensus.cc:482] T fe70cc275fab45d591f7bff25fbf59dd P a4ff3361f5784240bf29f7016bdaaa42 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:33:09.465910 13108 raft_consensus.cc:3037] T fe70cc275fab45d591f7bff25fbf59dd P a4ff3361f5784240bf29f7016bdaaa42 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:33:09.467027 13108 raft_consensus.cc:504] T fe70cc275fab45d591f7bff25fbf59dd P a4ff3361f5784240bf29f7016bdaaa42 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4ff3361f5784240bf29f7016bdaaa42" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 46029 } } peers { permanent_uuid: "62c0ffbc8d8d46fdb61ff8f094e1bb68" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 44813 } } peers { permanent_uuid: "34ede5db17834cfe86abe88e470bc41c" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 44069 } }
I20260807 18:33:09.467404 13108 leader_election.cc:288] T fe70cc275fab45d591f7bff25fbf59dd P a4ff3361f5784240bf29f7016bdaaa42 [CANDIDATE]: Term 1 election: Requested vote from peers 62c0ffbc8d8d46fdb61ff8f094e1bb68 (127.11.14.65:44813), 34ede5db17834cfe86abe88e470bc41c (127.11.14.66:44069)
I20260807 18:33:09.467852 12716 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe70cc275fab45d591f7bff25fbf59dd" candidate_uuid: "a4ff3361f5784240bf29f7016bdaaa42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62c0ffbc8d8d46fdb61ff8f094e1bb68"
I20260807 18:33:09.467852 12872 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe70cc275fab45d591f7bff25fbf59dd" candidate_uuid: "a4ff3361f5784240bf29f7016bdaaa42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34ede5db17834cfe86abe88e470bc41c"
I20260807 18:33:09.467981 12716 raft_consensus.cc:3037] T fe70cc275fab45d591f7bff25fbf59dd P 62c0ffbc8d8d46fdb61ff8f094e1bb68 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:33:09.467981 12872 raft_consensus.cc:3037] T fe70cc275fab45d591f7bff25fbf59dd P 34ede5db17834cfe86abe88e470bc41c [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:33:09.469446 12716 raft_consensus.cc:2454] T fe70cc275fab45d591f7bff25fbf59dd P 62c0ffbc8d8d46fdb61ff8f094e1bb68 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4ff3361f5784240bf29f7016bdaaa42 in term 1.
I20260807 18:33:09.469748 12952 leader_election.cc:302] T fe70cc275fab45d591f7bff25fbf59dd P a4ff3361f5784240bf29f7016bdaaa42 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 62c0ffbc8d8d46fdb61ff8f094e1bb68, a4ff3361f5784240bf29f7016bdaaa42; no voters:
I20260807 18:33:09.469970 13108 raft_consensus.cc:2781] T fe70cc275fab45d591f7bff25fbf59dd P a4ff3361f5784240bf29f7016bdaaa42 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:33:09.469939 12872 raft_consensus.cc:2454] T fe70cc275fab45d591f7bff25fbf59dd P 34ede5db17834cfe86abe88e470bc41c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4ff3361f5784240bf29f7016bdaaa42 in term 1.
I20260807 18:33:09.470245 13108 raft_consensus.cc:686] T fe70cc275fab45d591f7bff25fbf59dd P a4ff3361f5784240bf29f7016bdaaa42 [term 1 LEADER]: Becoming Leader. State: Replica: a4ff3361f5784240bf29f7016bdaaa42, State: Running, Role: LEADER
I20260807 18:33:09.470628 13108 consensus_queue.cc:237] T fe70cc275fab45d591f7bff25fbf59dd P a4ff3361f5784240bf29f7016bdaaa42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4ff3361f5784240bf29f7016bdaaa42" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 46029 } } peers { permanent_uuid: "62c0ffbc8d8d46fdb61ff8f094e1bb68" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 44813 } } peers { permanent_uuid: "34ede5db17834cfe86abe88e470bc41c" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 44069 } }
I20260807 18:33:09.473901 12543 catalog_manager.cc:5495] T fe70cc275fab45d591f7bff25fbf59dd P a4ff3361f5784240bf29f7016bdaaa42 reported cstate change: term changed from 0 to 1, leader changed from <none> to a4ff3361f5784240bf29f7016bdaaa42 (127.11.14.67). New cstate: current_term: 1 leader_uuid: "a4ff3361f5784240bf29f7016bdaaa42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4ff3361f5784240bf29f7016bdaaa42" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 46029 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "62c0ffbc8d8d46fdb61ff8f094e1bb68" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 44813 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34ede5db17834cfe86abe88e470bc41c" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 44069 } health_report { overall_health: UNKNOWN } } }
I20260807 18:33:09.496240 13096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:09.492685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57296 (local address 127.11.14.67:46029)
0807 18:33:09.492838 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:09.492847 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:09.492863 (+ 16us) server_negotiation.cc:407] Connection header received
0807 18:33:09.493037 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:09.493039 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:09.493093 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:09.493176 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:33:09.493425 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:09.493926 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:09.494765 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:09.494976 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:09.495049 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:09.495331 (+ 282us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:33:09.495835 (+ 504us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:33:09.495924 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:09.496008 (+ 84us) server_negotiation.cc:299] Negotiation successful
0807 18:33:09.496057 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260807 18:33:09.502951 12872 raft_consensus.cc:1261] T fe70cc275fab45d591f7bff25fbf59dd P 34ede5db17834cfe86abe88e470bc41c [term 1 FOLLOWER]: Refusing update from remote peer a4ff3361f5784240bf29f7016bdaaa42: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:33:09.503868 13108 consensus_queue.cc:1035] T fe70cc275fab45d591f7bff25fbf59dd P a4ff3361f5784240bf29f7016bdaaa42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34ede5db17834cfe86abe88e470bc41c" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 44069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:33:09.504679 12716 raft_consensus.cc:1261] T fe70cc275fab45d591f7bff25fbf59dd P 62c0ffbc8d8d46fdb61ff8f094e1bb68 [term 1 FOLLOWER]: Refusing update from remote peer a4ff3361f5784240bf29f7016bdaaa42: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:33:09.505270 13108 consensus_queue.cc:1035] T fe70cc275fab45d591f7bff25fbf59dd P a4ff3361f5784240bf29f7016bdaaa42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "62c0ffbc8d8d46fdb61ff8f094e1bb68" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 44813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:33:09.525352 12543 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53650:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:33:09.525619 12543 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53650:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:33:09.528436 12543 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a4c134b845384a56aa320c0f437b3512: Sending DeleteTablet for 3 replicas of tablet fe70cc275fab45d591f7bff25fbf59dd
I20260807 18:33:09.529354 13004 tablet_service.cc:1514] Processing DeleteTablet for tablet fe70cc275fab45d591f7bff25fbf59dd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:33:09 UTC) from {username='kudu', principal='kudu/127.11.14.126@KRBTEST.COM'} at 127.0.0.1:57286
I20260807 18:33:09.529346 12694 tablet_service.cc:1514] Processing DeleteTablet for tablet fe70cc275fab45d591f7bff25fbf59dd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:33:09 UTC) from {username='kudu', principal='kudu/127.11.14.126@KRBTEST.COM'} at 127.0.0.1:53726
I20260807 18:33:09.529431 12849 tablet_service.cc:1514] Processing DeleteTablet for tablet fe70cc275fab45d591f7bff25fbf59dd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:33:09 UTC) from {username='kudu', principal='kudu/127.11.14.126@KRBTEST.COM'} at 127.0.0.1:54876
I20260807 18:33:09.529971 13124 tablet_replica.cc:330] T fe70cc275fab45d591f7bff25fbf59dd P a4ff3361f5784240bf29f7016bdaaa42: stopping tablet replica
I20260807 18:33:09.530300 13124 raft_consensus.cc:2229] T fe70cc275fab45d591f7bff25fbf59dd P a4ff3361f5784240bf29f7016bdaaa42 [term 1 LEADER]: Raft consensus shutting down.
I20260807 18:33:09.530740 13124 raft_consensus.cc:2258] T fe70cc275fab45d591f7bff25fbf59dd P a4ff3361f5784240bf29f7016bdaaa42 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:33:09.531102 13126 tablet_replica.cc:330] T fe70cc275fab45d591f7bff25fbf59dd P 34ede5db17834cfe86abe88e470bc41c: stopping tablet replica
I20260807 18:33:09.531397 13126 raft_consensus.cc:2229] T fe70cc275fab45d591f7bff25fbf59dd P 34ede5db17834cfe86abe88e470bc41c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:33:09.531744 13126 raft_consensus.cc:2258] T fe70cc275fab45d591f7bff25fbf59dd P 34ede5db17834cfe86abe88e470bc41c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:33:09.533478 13126 ts_tablet_manager.cc:1905] T fe70cc275fab45d591f7bff25fbf59dd P 34ede5db17834cfe86abe88e470bc41c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:33:09.533500 13124 ts_tablet_manager.cc:1905] T fe70cc275fab45d591f7bff25fbf59dd P a4ff3361f5784240bf29f7016bdaaa42: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:33:09.533569 12543 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53650:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 18:33:09.534024 12543 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 18:33:09.536437 13126 ts_tablet_manager.cc:1918] T fe70cc275fab45d591f7bff25fbf59dd P 34ede5db17834cfe86abe88e470bc41c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:33:09.536542 13126 log.cc:1196] T fe70cc275fab45d591f7bff25fbf59dd P 34ede5db17834cfe86abe88e470bc41c: Deleting WAL directory at /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127579900523-11321-0/minicluster-data/ts-1/wal/wals/fe70cc275fab45d591f7bff25fbf59dd
I20260807 18:33:09.536896 13126 ts_tablet_manager.cc:1939] T fe70cc275fab45d591f7bff25fbf59dd P 34ede5db17834cfe86abe88e470bc41c: Deleting consensus metadata
I20260807 18:33:09.537055 13124 ts_tablet_manager.cc:1918] T fe70cc275fab45d591f7bff25fbf59dd P a4ff3361f5784240bf29f7016bdaaa42: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:33:09.537225 13124 log.cc:1196] T fe70cc275fab45d591f7bff25fbf59dd P a4ff3361f5784240bf29f7016bdaaa42: Deleting WAL directory at /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127579900523-11321-0/minicluster-data/ts-2/wal/wals/fe70cc275fab45d591f7bff25fbf59dd
I20260807 18:33:09.537695 13124 ts_tablet_manager.cc:1939] T fe70cc275fab45d591f7bff25fbf59dd P a4ff3361f5784240bf29f7016bdaaa42: Deleting consensus metadata
I20260807 18:33:09.538228 12525 catalog_manager.cc:4841] TS 34ede5db17834cfe86abe88e470bc41c (127.11.14.66:44069): tablet fe70cc275fab45d591f7bff25fbf59dd (table test-table [id=9be290545f254a99a1ea5a42abbf3c10]) successfully deleted
I20260807 18:33:09.538880 12527 catalog_manager.cc:4841] TS a4ff3361f5784240bf29f7016bdaaa42 (127.11.14.67:46029): tablet fe70cc275fab45d591f7bff25fbf59dd (table test-table [id=9be290545f254a99a1ea5a42abbf3c10]) successfully deleted
I20260807 18:33:09.539762 12694 tablet_service.cc:1467] Processing CreateTablet for tablet 9d65b116aa1e4027b215607ff2add626 (DEFAULT_TABLE table=test-table [id=b29e53041ae144b7a179ecc231ffa235]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:33:09.539876 12849 tablet_service.cc:1467] Processing CreateTablet for tablet 9d65b116aa1e4027b215607ff2add626 (DEFAULT_TABLE table=test-table [id=b29e53041ae144b7a179ecc231ffa235]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:33:09.540088 12694 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9d65b116aa1e4027b215607ff2add626. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:33:09.540109 12849 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9d65b116aa1e4027b215607ff2add626. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:33:09.540695 13125 tablet_replica.cc:330] T fe70cc275fab45d591f7bff25fbf59dd P 62c0ffbc8d8d46fdb61ff8f094e1bb68: stopping tablet replica
I20260807 18:33:09.541000 13125 raft_consensus.cc:2229] T fe70cc275fab45d591f7bff25fbf59dd P 62c0ffbc8d8d46fdb61ff8f094e1bb68 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:33:09.541296 13125 raft_consensus.cc:2258] T fe70cc275fab45d591f7bff25fbf59dd P 62c0ffbc8d8d46fdb61ff8f094e1bb68 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:33:09.542341 13099 tablet_bootstrap.cc:492] T 9d65b116aa1e4027b215607ff2add626 P 34ede5db17834cfe86abe88e470bc41c: Bootstrap starting.
I20260807 18:33:09.543329 13099 tablet_bootstrap.cc:654] T 9d65b116aa1e4027b215607ff2add626 P 34ede5db17834cfe86abe88e470bc41c: Neither blocks nor log segments found. Creating new log.
I20260807 18:33:09.543663 13125 ts_tablet_manager.cc:1905] T fe70cc275fab45d591f7bff25fbf59dd P 62c0ffbc8d8d46fdb61ff8f094e1bb68: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:33:09.543970 13100 tablet_bootstrap.cc:492] T 9d65b116aa1e4027b215607ff2add626 P 62c0ffbc8d8d46fdb61ff8f094e1bb68: Bootstrap starting.
I20260807 18:33:09.544916 13100 tablet_bootstrap.cc:654] T 9d65b116aa1e4027b215607ff2add626 P 62c0ffbc8d8d46fdb61ff8f094e1bb68: Neither blocks nor log segments found. Creating new log.
I20260807 18:33:09.546049 13099 tablet_bootstrap.cc:492] T 9d65b116aa1e4027b215607ff2add626 P 34ede5db17834cfe86abe88e470bc41c: No bootstrap required, opened a new log
I20260807 18:33:09.546135 13099 ts_tablet_manager.cc:1397] T 9d65b116aa1e4027b215607ff2add626 P 34ede5db17834cfe86abe88e470bc41c: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260807 18:33:09.546574 13125 ts_tablet_manager.cc:1918] T fe70cc275fab45d591f7bff25fbf59dd P 62c0ffbc8d8d46fdb61ff8f094e1bb68: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:33:09.546592 13099 raft_consensus.cc:348] T 9d65b116aa1e4027b215607ff2add626 P 34ede5db17834cfe86abe88e470bc41c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62c0ffbc8d8d46fdb61ff8f094e1bb68" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 44813 } } peers { permanent_uuid: "34ede5db17834cfe86abe88e470bc41c" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 44069 } } peers { permanent_uuid: "a4ff3361f5784240bf29f7016bdaaa42" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 46029 } }
I20260807 18:33:09.546684 13125 log.cc:1196] T fe70cc275fab45d591f7bff25fbf59dd P 62c0ffbc8d8d46fdb61ff8f094e1bb68: Deleting WAL directory at /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127579900523-11321-0/minicluster-data/ts-0/wal/wals/fe70cc275fab45d591f7bff25fbf59dd
I20260807 18:33:09.546705 13099 raft_consensus.cc:374] T 9d65b116aa1e4027b215607ff2add626 P 34ede5db17834cfe86abe88e470bc41c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:33:09.546754 13099 raft_consensus.cc:729] T 9d65b116aa1e4027b215607ff2add626 P 34ede5db17834cfe86abe88e470bc41c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34ede5db17834cfe86abe88e470bc41c, State: Initialized, Role: FOLLOWER
I20260807 18:33:09.546876 13099 consensus_queue.cc:260] T 9d65b116aa1e4027b215607ff2add626 P 34ede5db17834cfe86abe88e470bc41c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62c0ffbc8d8d46fdb61ff8f094e1bb68" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 44813 } } peers { permanent_uuid: "34ede5db17834cfe86abe88e470bc41c" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 44069 } } peers { permanent_uuid: "a4ff3361f5784240bf29f7016bdaaa42" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 46029 } }
I20260807 18:33:09.547073 13125 ts_tablet_manager.cc:1939] T fe70cc275fab45d591f7bff25fbf59dd P 62c0ffbc8d8d46fdb61ff8f094e1bb68: Deleting consensus metadata
I20260807 18:33:09.547138 13099 ts_tablet_manager.cc:1428] T 9d65b116aa1e4027b215607ff2add626 P 34ede5db17834cfe86abe88e470bc41c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 18:33:09.548125 13100 tablet_bootstrap.cc:492] T 9d65b116aa1e4027b215607ff2add626 P 62c0ffbc8d8d46fdb61ff8f094e1bb68: No bootstrap required, opened a new log
I20260807 18:33:09.548215 13100 ts_tablet_manager.cc:1397] T 9d65b116aa1e4027b215607ff2add626 P 62c0ffbc8d8d46fdb61ff8f094e1bb68: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260807 18:33:09.548358 12525 catalog_manager.cc:4841] TS 62c0ffbc8d8d46fdb61ff8f094e1bb68 (127.11.14.65:44813): tablet fe70cc275fab45d591f7bff25fbf59dd (table test-table [id=9be290545f254a99a1ea5a42abbf3c10]) successfully deleted
I20260807 18:33:09.548933 13004 tablet_service.cc:1467] Processing CreateTablet for tablet 9d65b116aa1e4027b215607ff2add626 (DEFAULT_TABLE table=test-table [id=b29e53041ae144b7a179ecc231ffa235]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:33:09.549047 13100 raft_consensus.cc:348] T 9d65b116aa1e4027b215607ff2add626 P 62c0ffbc8d8d46fdb61ff8f094e1bb68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62c0ffbc8d8d46fdb61ff8f094e1bb68" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 44813 } } peers { permanent_uuid: "34ede5db17834cfe86abe88e470bc41c" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 44069 } } peers { permanent_uuid: "a4ff3361f5784240bf29f7016bdaaa42" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 46029 } }
I20260807 18:33:09.549157 13100 raft_consensus.cc:374] T 9d65b116aa1e4027b215607ff2add626 P 62c0ffbc8d8d46fdb61ff8f094e1bb68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:33:09.549197 13100 raft_consensus.cc:729] T 9d65b116aa1e4027b215607ff2add626 P 62c0ffbc8d8d46fdb61ff8f094e1bb68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 62c0ffbc8d8d46fdb61ff8f094e1bb68, State: Initialized, Role: FOLLOWER
I20260807 18:33:09.549329 13100 consensus_queue.cc:260] T 9d65b116aa1e4027b215607ff2add626 P 62c0ffbc8d8d46fdb61ff8f094e1bb68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62c0ffbc8d8d46fdb61ff8f094e1bb68" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 44813 } } peers { permanent_uuid: "34ede5db17834cfe86abe88e470bc41c" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 44069 } } peers { permanent_uuid: "a4ff3361f5784240bf29f7016bdaaa42" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 46029 } }
I20260807 18:33:09.549501 13004 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9d65b116aa1e4027b215607ff2add626. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:33:09.550014 13100 ts_tablet_manager.cc:1428] T 9d65b116aa1e4027b215607ff2add626 P 62c0ffbc8d8d46fdb61ff8f094e1bb68: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260807 18:33:09.552722 13101 tablet_bootstrap.cc:492] T 9d65b116aa1e4027b215607ff2add626 P a4ff3361f5784240bf29f7016bdaaa42: Bootstrap starting.
W20260807 18:33:09.552997 12774 tablet.cc:2369] T 9d65b116aa1e4027b215607ff2add626 P 62c0ffbc8d8d46fdb61ff8f094e1bb68: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:33:09.553751 13101 tablet_bootstrap.cc:654] T 9d65b116aa1e4027b215607ff2add626 P a4ff3361f5784240bf29f7016bdaaa42: Neither blocks nor log segments found. Creating new log.
I20260807 18:33:09.555060 13101 tablet_bootstrap.cc:492] T 9d65b116aa1e4027b215607ff2add626 P a4ff3361f5784240bf29f7016bdaaa42: No bootstrap required, opened a new log
I20260807 18:33:09.555147 13101 ts_tablet_manager.cc:1397] T 9d65b116aa1e4027b215607ff2add626 P a4ff3361f5784240bf29f7016bdaaa42: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260807 18:33:09.555657 13101 raft_consensus.cc:348] T 9d65b116aa1e4027b215607ff2add626 P a4ff3361f5784240bf29f7016bdaaa42 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62c0ffbc8d8d46fdb61ff8f094e1bb68" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 44813 } } peers { permanent_uuid: "34ede5db17834cfe86abe88e470bc41c" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 44069 } } peers { permanent_uuid: "a4ff3361f5784240bf29f7016bdaaa42" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 46029 } }
I20260807 18:33:09.555780 13101 raft_consensus.cc:374] T 9d65b116aa1e4027b215607ff2add626 P a4ff3361f5784240bf29f7016bdaaa42 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:33:09.555814 13101 raft_consensus.cc:729] T 9d65b116aa1e4027b215607ff2add626 P a4ff3361f5784240bf29f7016bdaaa42 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4ff3361f5784240bf29f7016bdaaa42, State: Initialized, Role: FOLLOWER
I20260807 18:33:09.555943 13101 consensus_queue.cc:260] T 9d65b116aa1e4027b215607ff2add626 P a4ff3361f5784240bf29f7016bdaaa42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62c0ffbc8d8d46fdb61ff8f094e1bb68" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 44813 } } peers { permanent_uuid: "34ede5db17834cfe86abe88e470bc41c" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 44069 } } peers { permanent_uuid: "a4ff3361f5784240bf29f7016bdaaa42" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 46029 } }
I20260807 18:33:09.556195 13101 ts_tablet_manager.cc:1428] T 9d65b116aa1e4027b215607ff2add626 P a4ff3361f5784240bf29f7016bdaaa42: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
W20260807 18:33:09.562000 12924 tablet.cc:2369] T 9d65b116aa1e4027b215607ff2add626 P 34ede5db17834cfe86abe88e470bc41c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:33:09.619859 13106 raft_consensus.cc:482] T 9d65b116aa1e4027b215607ff2add626 P 34ede5db17834cfe86abe88e470bc41c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:33:09.620033 13106 raft_consensus.cc:504] T 9d65b116aa1e4027b215607ff2add626 P 34ede5db17834cfe86abe88e470bc41c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62c0ffbc8d8d46fdb61ff8f094e1bb68" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 44813 } } peers { permanent_uuid: "34ede5db17834cfe86abe88e470bc41c" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 44069 } } peers { permanent_uuid: "a4ff3361f5784240bf29f7016bdaaa42" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 46029 } }
I20260807 18:33:09.621299 13106 leader_election.cc:288] T 9d65b116aa1e4027b215607ff2add626 P 34ede5db17834cfe86abe88e470bc41c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 62c0ffbc8d8d46fdb61ff8f094e1bb68 (127.11.14.65:44813), a4ff3361f5784240bf29f7016bdaaa42 (127.11.14.67:46029)
I20260807 18:33:09.626250 13096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:09.621667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.66:38863 (local address 127.11.14.67:46029)
0807 18:33:09.621821 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:09.621826 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:09.622126 (+ 300us) server_negotiation.cc:407] Connection header received
0807 18:33:09.622317 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:09.622320 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:09.622366 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:09.622431 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:33:09.622792 (+ 361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:09.623272 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:09.625101 (+ 1829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:09.625686 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:09.625828 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:09.625887 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:09.626057 (+ 170us) server_negotiation.cc:299] Negotiation successful
0807 18:33:09.626135 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260807 18:33:09.626266 13132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:09.621519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.67:46029 (local address 127.11.14.66:38863)
0807 18:33:09.622023 (+ 504us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:09.622034 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:09.622211 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:09.622517 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:09.622521 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:09.622533 (+ 12us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:33:09.622671 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:09.622678 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:09.623431 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:09.623438 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:09.624962 (+ 1524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:09.624970 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:09.625824 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:09.625827 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:09.625914 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:09.625935 (+ 21us) client_negotiation.cc:769] Sending connection context
0807 18:33:09.626032 (+ 97us) client_negotiation.cc:240] Negotiation successful
0807 18:33:09.626111 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":452,"mutex_wait_us":321,"thread_start_us":68,"threads_started":1}
I20260807 18:33:09.627177 13095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:09.621745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.66:44587 (local address 127.11.14.65:44813)
0807 18:33:09.621906 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:09.621912 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:09.622094 (+ 182us) server_negotiation.cc:407] Connection header received
0807 18:33:09.622222 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:09.622225 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:09.622281 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:09.622384 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:33:09.622681 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:09.623466 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:09.625948 (+ 2482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:09.626590 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:09.626718 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:09.626780 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:09.626965 (+ 185us) server_negotiation.cc:299] Negotiation successful
0807 18:33:09.627053 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260807 18:33:09.627625 13131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:09.621519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.65:44813 (local address 127.11.14.66:44587)
0807 18:33:09.621979 (+ 460us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:09.621995 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:09.622111 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:09.622396 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:09.622399 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:09.622414 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:33:09.622559 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:09.622568 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:09.623601 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:09.623604 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:09.624958 (+ 1354us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:09.624970 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:09.626712 (+ 1742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:09.626715 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:09.626791 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:09.626810 (+ 19us) client_negotiation.cc:769] Sending connection context
0807 18:33:09.627327 (+ 517us) client_negotiation.cc:240] Negotiation successful
0807 18:33:09.627431 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":388,"mutex_wait_us":110,"thread_start_us":75,"threads_started":1}
I20260807 18:33:09.627682 13026 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d65b116aa1e4027b215607ff2add626" candidate_uuid: "34ede5db17834cfe86abe88e470bc41c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4ff3361f5784240bf29f7016bdaaa42" is_pre_election: true
I20260807 18:33:09.627969 13026 raft_consensus.cc:2454] T 9d65b116aa1e4027b215607ff2add626 P a4ff3361f5784240bf29f7016bdaaa42 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34ede5db17834cfe86abe88e470bc41c in term 0.
I20260807 18:33:09.628139 12716 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d65b116aa1e4027b215607ff2add626" candidate_uuid: "34ede5db17834cfe86abe88e470bc41c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62c0ffbc8d8d46fdb61ff8f094e1bb68" is_pre_election: true
I20260807 18:33:09.628314 12716 raft_consensus.cc:2454] T 9d65b116aa1e4027b215607ff2add626 P 62c0ffbc8d8d46fdb61ff8f094e1bb68 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34ede5db17834cfe86abe88e470bc41c in term 0.
I20260807 18:33:09.628423 12799 leader_election.cc:302] T 9d65b116aa1e4027b215607ff2add626 P 34ede5db17834cfe86abe88e470bc41c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34ede5db17834cfe86abe88e470bc41c, a4ff3361f5784240bf29f7016bdaaa42; no voters:
I20260807 18:33:09.628707 13106 raft_consensus.cc:2781] T 9d65b116aa1e4027b215607ff2add626 P 34ede5db17834cfe86abe88e470bc41c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:33:09.628779 13106 raft_consensus.cc:482] T 9d65b116aa1e4027b215607ff2add626 P 34ede5db17834cfe86abe88e470bc41c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:33:09.628818 13106 raft_consensus.cc:3037] T 9d65b116aa1e4027b215607ff2add626 P 34ede5db17834cfe86abe88e470bc41c [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:33:09.630162 13106 raft_consensus.cc:504] T 9d65b116aa1e4027b215607ff2add626 P 34ede5db17834cfe86abe88e470bc41c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62c0ffbc8d8d46fdb61ff8f094e1bb68" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 44813 } } peers { permanent_uuid: "34ede5db17834cfe86abe88e470bc41c" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 44069 } } peers { permanent_uuid: "a4ff3361f5784240bf29f7016bdaaa42" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 46029 } }
I20260807 18:33:09.630540 13106 leader_election.cc:288] T 9d65b116aa1e4027b215607ff2add626 P 34ede5db17834cfe86abe88e470bc41c [CANDIDATE]: Term 1 election: Requested vote from peers 62c0ffbc8d8d46fdb61ff8f094e1bb68 (127.11.14.65:44813), a4ff3361f5784240bf29f7016bdaaa42 (127.11.14.67:46029)
I20260807 18:33:09.630930 13026 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d65b116aa1e4027b215607ff2add626" candidate_uuid: "34ede5db17834cfe86abe88e470bc41c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4ff3361f5784240bf29f7016bdaaa42"
I20260807 18:33:09.630945 12716 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d65b116aa1e4027b215607ff2add626" candidate_uuid: "34ede5db17834cfe86abe88e470bc41c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62c0ffbc8d8d46fdb61ff8f094e1bb68"
I20260807 18:33:09.631067 12716 raft_consensus.cc:3037] T 9d65b116aa1e4027b215607ff2add626 P 62c0ffbc8d8d46fdb61ff8f094e1bb68 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:33:09.631071 13026 raft_consensus.cc:3037] T 9d65b116aa1e4027b215607ff2add626 P a4ff3361f5784240bf29f7016bdaaa42 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:33:09.632253 13026 raft_consensus.cc:2454] T 9d65b116aa1e4027b215607ff2add626 P a4ff3361f5784240bf29f7016bdaaa42 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34ede5db17834cfe86abe88e470bc41c in term 1.
I20260807 18:33:09.632341 12716 raft_consensus.cc:2454] T 9d65b116aa1e4027b215607ff2add626 P 62c0ffbc8d8d46fdb61ff8f094e1bb68 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34ede5db17834cfe86abe88e470bc41c in term 1.
I20260807 18:33:09.632547 12799 leader_election.cc:302] T 9d65b116aa1e4027b215607ff2add626 P 34ede5db17834cfe86abe88e470bc41c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34ede5db17834cfe86abe88e470bc41c, a4ff3361f5784240bf29f7016bdaaa42; no voters:
I20260807 18:33:09.632709 13106 raft_consensus.cc:2781] T 9d65b116aa1e4027b215607ff2add626 P 34ede5db17834cfe86abe88e470bc41c [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:33:09.632953 13106 raft_consensus.cc:686] T 9d65b116aa1e4027b215607ff2add626 P 34ede5db17834cfe86abe88e470bc41c [term 1 LEADER]: Becoming Leader. State: Replica: 34ede5db17834cfe86abe88e470bc41c, State: Running, Role: LEADER
I20260807 18:33:09.633200 13106 consensus_queue.cc:237] T 9d65b116aa1e4027b215607ff2add626 P 34ede5db17834cfe86abe88e470bc41c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62c0ffbc8d8d46fdb61ff8f094e1bb68" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 44813 } } peers { permanent_uuid: "34ede5db17834cfe86abe88e470bc41c" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 44069 } } peers { permanent_uuid: "a4ff3361f5784240bf29f7016bdaaa42" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 46029 } }
I20260807 18:33:09.635499 12542 catalog_manager.cc:5495] T 9d65b116aa1e4027b215607ff2add626 P 34ede5db17834cfe86abe88e470bc41c reported cstate change: term changed from 0 to 1, leader changed from <none> to 34ede5db17834cfe86abe88e470bc41c (127.11.14.66). New cstate: current_term: 1 leader_uuid: "34ede5db17834cfe86abe88e470bc41c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62c0ffbc8d8d46fdb61ff8f094e1bb68" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 44813 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34ede5db17834cfe86abe88e470bc41c" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 44069 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4ff3361f5784240bf29f7016bdaaa42" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 46029 } health_report { overall_health: UNKNOWN } } }
Aug 07 18:33:09 dist-test-slave-6k22 krb5kdc[12492](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127588, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.126@KRBTEST.COM for kudu/127.11.14.126@KRBTEST.COM
I20260807 18:33:09.674839 13143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:09.665452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:34397 (local address 127.0.0.1:53652)
0807 18:33:09.665792 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:09.665809 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:09.665943 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:09.666219 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:09.666222 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:09.666797 (+ 575us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:33:09.666925 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:09.666933 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:09.667711 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:09.667715 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:09.668580 (+ 865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:09.668591 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:09.669028 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:09.669031 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:09.669127 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:09.669805 (+ 678us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:33:09.669817 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:33:09.671587 (+ 1770us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:33:09.673179 (+ 1592us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:33:09.673191 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:33:09.673198 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:33:09.673437 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:33:09.673719 (+ 282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:33:09.673722 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:33:09.673724 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:33:09.673807 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:33:09.674200 (+ 393us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:33:09.674209 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:33:09.674323 (+ 114us) client_negotiation.cc:769] Sending connection context
0807 18:33:09.674440 (+ 117us) client_negotiation.cc:240] Negotiation successful
0807 18:33:09.674602 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":104,"threads_started":1}
I20260807 18:33:09.674844 12597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:09.665528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53652 (local address 127.11.14.126:34397)
0807 18:33:09.665678 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:09.665682 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:09.665900 (+ 218us) server_negotiation.cc:407] Connection header received
0807 18:33:09.666042 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:09.666045 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:09.666084 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:09.666156 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:33:09.667050 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:09.667599 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:09.668708 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:09.668917 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:09.668993 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:09.671713 (+ 2720us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:33:09.671717 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:33:09.671719 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:33:09.671742 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:33:09.673067 (+ 1325us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:09.673536 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:09.673541 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:09.673543 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:09.673620 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:09.673901 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:09.673904 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:09.673906 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:09.674083 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:33:09.674190 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:09.674500 (+ 310us) server_negotiation.cc:299] Negotiation successful
0807 18:33:09.674658 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":58,"server-negotiator.queue_time_us":63}
I20260807 18:33:09.679376 12542 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.11.14.126@KRBTEST.COM'} at 127.0.0.1:53652:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260807 18:33:09.680182 12542 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 18:33:09.685395 12694 tablet_service.cc:1467] Processing CreateTablet for tablet 72aeff6bf23645a88525429b8c145be1 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=aa38daec956743998a74d6aaba78c413]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 18:33:09.685453 12849 tablet_service.cc:1467] Processing CreateTablet for tablet 72aeff6bf23645a88525429b8c145be1 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=aa38daec956743998a74d6aaba78c413]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 18:33:09.685844 12694 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 72aeff6bf23645a88525429b8c145be1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:33:09.685850 12849 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 72aeff6bf23645a88525429b8c145be1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:33:09.685395 13004 tablet_service.cc:1467] Processing CreateTablet for tablet 72aeff6bf23645a88525429b8c145be1 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=aa38daec956743998a74d6aaba78c413]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 18:33:09.688414 13004 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 72aeff6bf23645a88525429b8c145be1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:33:09.689047 13099 tablet_bootstrap.cc:492] T 72aeff6bf23645a88525429b8c145be1 P 34ede5db17834cfe86abe88e470bc41c: Bootstrap starting.
I20260807 18:33:09.690109 13099 tablet_bootstrap.cc:654] T 72aeff6bf23645a88525429b8c145be1 P 34ede5db17834cfe86abe88e470bc41c: Neither blocks nor log segments found. Creating new log.
I20260807 18:33:09.691764 13100 tablet_bootstrap.cc:492] T 72aeff6bf23645a88525429b8c145be1 P 62c0ffbc8d8d46fdb61ff8f094e1bb68: Bootstrap starting.
I20260807 18:33:09.692807 13100 tablet_bootstrap.cc:654] T 72aeff6bf23645a88525429b8c145be1 P 62c0ffbc8d8d46fdb61ff8f094e1bb68: Neither blocks nor log segments found. Creating new log.
I20260807 18:33:09.692912 13099 tablet_bootstrap.cc:492] T 72aeff6bf23645a88525429b8c145be1 P 34ede5db17834cfe86abe88e470bc41c: No bootstrap required, opened a new log
I20260807 18:33:09.693003 13099 ts_tablet_manager.cc:1397] T 72aeff6bf23645a88525429b8c145be1 P 34ede5db17834cfe86abe88e470bc41c: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260807 18:33:09.693416 13099 raft_consensus.cc:348] T 72aeff6bf23645a88525429b8c145be1 P 34ede5db17834cfe86abe88e470bc41c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62c0ffbc8d8d46fdb61ff8f094e1bb68" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 44813 } } peers { permanent_uuid: "34ede5db17834cfe86abe88e470bc41c" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 44069 } } peers { permanent_uuid: "a4ff3361f5784240bf29f7016bdaaa42" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 46029 } }
I20260807 18:33:09.693534 13099 raft_consensus.cc:374] T 72aeff6bf23645a88525429b8c145be1 P 34ede5db17834cfe86abe88e470bc41c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:33:09.693583 13099 raft_consensus.cc:729] T 72aeff6bf23645a88525429b8c145be1 P 34ede5db17834cfe86abe88e470bc41c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34ede5db17834cfe86abe88e470bc41c, State: Initialized, Role: FOLLOWER
I20260807 18:33:09.693698 13099 consensus_queue.cc:260] T 72aeff6bf23645a88525429b8c145be1 P 34ede5db17834cfe86abe88e470bc41c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62c0ffbc8d8d46fdb61ff8f094e1bb68" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 44813 } } peers { permanent_uuid: "34ede5db17834cfe86abe88e470bc41c" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 44069 } } peers { permanent_uuid: "a4ff3361f5784240bf29f7016bdaaa42" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 46029 } }
I20260807 18:33:09.694018 13106 tablet_replica.cc:439] T 72aeff6bf23645a88525429b8c145be1 P 34ede5db17834cfe86abe88e470bc41c: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62c0ffbc8d8d46fdb61ff8f094e1bb68" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 44813 } } peers { permanent_uuid: "34ede5db17834cfe86abe88e470bc41c" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 44069 } } peers { permanent_uuid: "a4ff3361f5784240bf29f7016bdaaa42" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 46029 } } }
I20260807 18:33:09.694146 13106 tablet_replica.cc:442] T 72aeff6bf23645a88525429b8c145be1 P 34ede5db17834cfe86abe88e470bc41c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 18:33:09.694339 13100 tablet_bootstrap.cc:492] T 72aeff6bf23645a88525429b8c145be1 P 62c0ffbc8d8d46fdb61ff8f094e1bb68: No bootstrap required, opened a new log
I20260807 18:33:09.694444 13100 ts_tablet_manager.cc:1397] T 72aeff6bf23645a88525429b8c145be1 P 62c0ffbc8d8d46fdb61ff8f094e1bb68: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260807 18:33:09.694697 13099 ts_tablet_manager.cc:1428] T 72aeff6bf23645a88525429b8c145be1 P 34ede5db17834cfe86abe88e470bc41c: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260807 18:33:09.694937 13100 raft_consensus.cc:348] T 72aeff6bf23645a88525429b8c145be1 P 62c0ffbc8d8d46fdb61ff8f094e1bb68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62c0ffbc8d8d46fdb61ff8f094e1bb68" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 44813 } } peers { permanent_uuid: "34ede5db17834cfe86abe88e470bc41c" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 44069 } } peers { permanent_uuid: "a4ff3361f5784240bf29f7016bdaaa42" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 46029 } }
I20260807 18:33:09.695063 13100 raft_consensus.cc:374] T 72aeff6bf23645a88525429b8c145be1 P 62c0ffbc8d8d46fdb61ff8f094e1bb68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:33:09.695103 13100 raft_consensus.cc:729] T 72aeff6bf23645a88525429b8c145be1 P 62c0ffbc8d8d46fdb61ff8f094e1bb68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 62c0ffbc8d8d46fdb61ff8f094e1bb68, State: Initialized, Role: FOLLOWER
I20260807 18:33:09.695242 13100 consensus_queue.cc:260] T 72aeff6bf23645a88525429b8c145be1 P 62c0ffbc8d8d46fdb61ff8f094e1bb68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62c0ffbc8d8d46fdb61ff8f094e1bb68" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 44813 } } peers { permanent_uuid: "34ede5db17834cfe86abe88e470bc41c" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 44069 } } peers { permanent_uuid: "a4ff3361f5784240bf29f7016bdaaa42" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 46029 } }
I20260807 18:33:09.695510 13109 tablet_replica.cc:439] T 72aeff6bf23645a88525429b8c145be1 P 62c0ffbc8d8d46fdb61ff8f094e1bb68: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62c0ffbc8d8d46fdb61ff8f094e1bb68" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 44813 } } peers { permanent_uuid: "34ede5db17834cfe86abe88e470bc41c" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 44069 } } peers { permanent_uuid: "a4ff3361f5784240bf29f7016bdaaa42" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 46029 } } }
I20260807 18:33:09.695669 13109 tablet_replica.cc:442] T 72aeff6bf23645a88525429b8c145be1 P 62c0ffbc8d8d46fdb61ff8f094e1bb68: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 18:33:09.695927 13100 ts_tablet_manager.cc:1428] T 72aeff6bf23645a88525429b8c145be1 P 62c0ffbc8d8d46fdb61ff8f094e1bb68: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 18:33:09.696405 13101 tablet_bootstrap.cc:492] T 72aeff6bf23645a88525429b8c145be1 P a4ff3361f5784240bf29f7016bdaaa42: Bootstrap starting.
I20260807 18:33:09.697304 13101 tablet_bootstrap.cc:654] T 72aeff6bf23645a88525429b8c145be1 P a4ff3361f5784240bf29f7016bdaaa42: Neither blocks nor log segments found. Creating new log.
I20260807 18:33:09.698529 13101 tablet_bootstrap.cc:492] T 72aeff6bf23645a88525429b8c145be1 P a4ff3361f5784240bf29f7016bdaaa42: No bootstrap required, opened a new log
I20260807 18:33:09.698602 13101 ts_tablet_manager.cc:1397] T 72aeff6bf23645a88525429b8c145be1 P a4ff3361f5784240bf29f7016bdaaa42: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 18:33:09.698995 13101 raft_consensus.cc:348] T 72aeff6bf23645a88525429b8c145be1 P a4ff3361f5784240bf29f7016bdaaa42 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62c0ffbc8d8d46fdb61ff8f094e1bb68" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 44813 } } peers { permanent_uuid: "34ede5db17834cfe86abe88e470bc41c" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 44069 } } peers { permanent_uuid: "a4ff3361f5784240bf29f7016bdaaa42" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 46029 } }
I20260807 18:33:09.699117 13101 raft_consensus.cc:374] T 72aeff6bf23645a88525429b8c145be1 P a4ff3361f5784240bf29f7016bdaaa42 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:33:09.699141 13101 raft_consensus.cc:729] T 72aeff6bf23645a88525429b8c145be1 P a4ff3361f5784240bf29f7016bdaaa42 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4ff3361f5784240bf29f7016bdaaa42, State: Initialized, Role: FOLLOWER
I20260807 18:33:09.699278 13101 consensus_queue.cc:260] T 72aeff6bf23645a88525429b8c145be1 P a4ff3361f5784240bf29f7016bdaaa42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62c0ffbc8d8d46fdb61ff8f094e1bb68" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 44813 } } peers { permanent_uuid: "34ede5db17834cfe86abe88e470bc41c" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 44069 } } peers { permanent_uuid: "a4ff3361f5784240bf29f7016bdaaa42" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 46029 } }
I20260807 18:33:09.699632 13101 ts_tablet_manager.cc:1428] T 72aeff6bf23645a88525429b8c145be1 P a4ff3361f5784240bf29f7016bdaaa42: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 18:33:09.699594 13108 tablet_replica.cc:439] T 72aeff6bf23645a88525429b8c145be1 P a4ff3361f5784240bf29f7016bdaaa42: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62c0ffbc8d8d46fdb61ff8f094e1bb68" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 44813 } } peers { permanent_uuid: "34ede5db17834cfe86abe88e470bc41c" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 44069 } } peers { permanent_uuid: "a4ff3361f5784240bf29f7016bdaaa42" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 46029 } } }
I20260807 18:33:09.699792 13108 tablet_replica.cc:442] T 72aeff6bf23645a88525429b8c145be1 P a4ff3361f5784240bf29f7016bdaaa42: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 18:33:09.794883 13108 raft_consensus.cc:482] T 72aeff6bf23645a88525429b8c145be1 P a4ff3361f5784240bf29f7016bdaaa42 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:33:09.795053 13108 raft_consensus.cc:504] T 72aeff6bf23645a88525429b8c145be1 P a4ff3361f5784240bf29f7016bdaaa42 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62c0ffbc8d8d46fdb61ff8f094e1bb68" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 44813 } } peers { permanent_uuid: "34ede5db17834cfe86abe88e470bc41c" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 44069 } } peers { permanent_uuid: "a4ff3361f5784240bf29f7016bdaaa42" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 46029 } }
I20260807 18:33:09.795462 13108 leader_election.cc:288] T 72aeff6bf23645a88525429b8c145be1 P a4ff3361f5784240bf29f7016bdaaa42 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 62c0ffbc8d8d46fdb61ff8f094e1bb68 (127.11.14.65:44813), 34ede5db17834cfe86abe88e470bc41c (127.11.14.66:44069)
I20260807 18:33:09.796000 12716 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "72aeff6bf23645a88525429b8c145be1" candidate_uuid: "a4ff3361f5784240bf29f7016bdaaa42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62c0ffbc8d8d46fdb61ff8f094e1bb68" is_pre_election: true
I20260807 18:33:09.796079 12872 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "72aeff6bf23645a88525429b8c145be1" candidate_uuid: "a4ff3361f5784240bf29f7016bdaaa42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34ede5db17834cfe86abe88e470bc41c" is_pre_election: true
I20260807 18:33:09.796165 12716 raft_consensus.cc:2454] T 72aeff6bf23645a88525429b8c145be1 P 62c0ffbc8d8d46fdb61ff8f094e1bb68 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4ff3361f5784240bf29f7016bdaaa42 in term 0.
I20260807 18:33:09.796290 12872 raft_consensus.cc:2454] T 72aeff6bf23645a88525429b8c145be1 P 34ede5db17834cfe86abe88e470bc41c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4ff3361f5784240bf29f7016bdaaa42 in term 0.
I20260807 18:33:09.796535 12952 leader_election.cc:302] T 72aeff6bf23645a88525429b8c145be1 P a4ff3361f5784240bf29f7016bdaaa42 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 62c0ffbc8d8d46fdb61ff8f094e1bb68, a4ff3361f5784240bf29f7016bdaaa42; no voters:
I20260807 18:33:09.796732 13108 raft_consensus.cc:2781] T 72aeff6bf23645a88525429b8c145be1 P a4ff3361f5784240bf29f7016bdaaa42 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:33:09.796828 13108 raft_consensus.cc:482] T 72aeff6bf23645a88525429b8c145be1 P a4ff3361f5784240bf29f7016bdaaa42 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:33:09.796888 13108 raft_consensus.cc:3037] T 72aeff6bf23645a88525429b8c145be1 P a4ff3361f5784240bf29f7016bdaaa42 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:33:09.797922 13108 raft_consensus.cc:504] T 72aeff6bf23645a88525429b8c145be1 P a4ff3361f5784240bf29f7016bdaaa42 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62c0ffbc8d8d46fdb61ff8f094e1bb68" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 44813 } } peers { permanent_uuid: "34ede5db17834cfe86abe88e470bc41c" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 44069 } } peers { permanent_uuid: "a4ff3361f5784240bf29f7016bdaaa42" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 46029 } }
I20260807 18:33:09.798255 13108 leader_election.cc:288] T 72aeff6bf23645a88525429b8c145be1 P a4ff3361f5784240bf29f7016bdaaa42 [CANDIDATE]: Term 1 election: Requested vote from peers 62c0ffbc8d8d46fdb61ff8f094e1bb68 (127.11.14.65:44813), 34ede5db17834cfe86abe88e470bc41c (127.11.14.66:44069)
I20260807 18:33:09.798609 12716 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "72aeff6bf23645a88525429b8c145be1" candidate_uuid: "a4ff3361f5784240bf29f7016bdaaa42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62c0ffbc8d8d46fdb61ff8f094e1bb68"
I20260807 18:33:09.798710 12716 raft_consensus.cc:3037] T 72aeff6bf23645a88525429b8c145be1 P 62c0ffbc8d8d46fdb61ff8f094e1bb68 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:33:09.798696 12872 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "72aeff6bf23645a88525429b8c145be1" candidate_uuid: "a4ff3361f5784240bf29f7016bdaaa42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34ede5db17834cfe86abe88e470bc41c"
I20260807 18:33:09.798815 12872 raft_consensus.cc:3037] T 72aeff6bf23645a88525429b8c145be1 P 34ede5db17834cfe86abe88e470bc41c [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:33:09.799783 12716 raft_consensus.cc:2454] T 72aeff6bf23645a88525429b8c145be1 P 62c0ffbc8d8d46fdb61ff8f094e1bb68 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4ff3361f5784240bf29f7016bdaaa42 in term 1.
I20260807 18:33:09.800062 12872 raft_consensus.cc:2454] T 72aeff6bf23645a88525429b8c145be1 P 34ede5db17834cfe86abe88e470bc41c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4ff3361f5784240bf29f7016bdaaa42 in term 1.
I20260807 18:33:09.800128 12952 leader_election.cc:302] T 72aeff6bf23645a88525429b8c145be1 P a4ff3361f5784240bf29f7016bdaaa42 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 62c0ffbc8d8d46fdb61ff8f094e1bb68, a4ff3361f5784240bf29f7016bdaaa42; no voters:
W20260807 18:33:09.800424 13077 tablet.cc:2369] T 72aeff6bf23645a88525429b8c145be1 P a4ff3361f5784240bf29f7016bdaaa42: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:33:09.801040 13108 raft_consensus.cc:2781] T 72aeff6bf23645a88525429b8c145be1 P a4ff3361f5784240bf29f7016bdaaa42 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:33:09.801133 13108 raft_consensus.cc:686] T 72aeff6bf23645a88525429b8c145be1 P a4ff3361f5784240bf29f7016bdaaa42 [term 1 LEADER]: Becoming Leader. State: Replica: a4ff3361f5784240bf29f7016bdaaa42, State: Running, Role: LEADER
I20260807 18:33:09.801282 13108 consensus_queue.cc:237] T 72aeff6bf23645a88525429b8c145be1 P a4ff3361f5784240bf29f7016bdaaa42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62c0ffbc8d8d46fdb61ff8f094e1bb68" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 44813 } } peers { permanent_uuid: "34ede5db17834cfe86abe88e470bc41c" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 44069 } } peers { permanent_uuid: "a4ff3361f5784240bf29f7016bdaaa42" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 46029 } }
I20260807 18:33:09.802011 13111 tablet_replica.cc:439] T 72aeff6bf23645a88525429b8c145be1 P a4ff3361f5784240bf29f7016bdaaa42: TxnStatusTablet state changed. Reason: New leader a4ff3361f5784240bf29f7016bdaaa42. Latest consensus state: current_term: 1 leader_uuid: "a4ff3361f5784240bf29f7016bdaaa42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62c0ffbc8d8d46fdb61ff8f094e1bb68" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 44813 } } peers { permanent_uuid: "34ede5db17834cfe86abe88e470bc41c" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 44069 } } peers { permanent_uuid: "a4ff3361f5784240bf29f7016bdaaa42" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 46029 } } }
I20260807 18:33:09.802139 13111 tablet_replica.cc:442] T 72aeff6bf23645a88525429b8c145be1 P a4ff3361f5784240bf29f7016bdaaa42: This TxnStatusTablet replica's current role is: LEADER
I20260807 18:33:09.803155 12542 catalog_manager.cc:5495] T 72aeff6bf23645a88525429b8c145be1 P a4ff3361f5784240bf29f7016bdaaa42 reported cstate change: term changed from 0 to 1, leader changed from <none> to a4ff3361f5784240bf29f7016bdaaa42 (127.11.14.67). New cstate: current_term: 1 leader_uuid: "a4ff3361f5784240bf29f7016bdaaa42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62c0ffbc8d8d46fdb61ff8f094e1bb68" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 44813 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34ede5db17834cfe86abe88e470bc41c" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 44069 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4ff3361f5784240bf29f7016bdaaa42" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 46029 } health_report { overall_health: HEALTHY } } }
I20260807 18:33:09.807780 13149 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260807 18:33:09.807993 13149 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260807 18:33:09.823091 13096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:09.819473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57306 (local address 127.11.14.67:46029)
0807 18:33:09.819616 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:09.819621 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:09.819758 (+ 137us) server_negotiation.cc:407] Connection header received
0807 18:33:09.819927 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:09.819929 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:09.819973 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:09.820039 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:33:09.820328 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:09.820834 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:09.822065 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:09.822278 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:09.822354 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:09.822670 (+ 316us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:33:09.822746 (+ 76us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:33:09.822798 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:09.822936 (+ 138us) server_negotiation.cc:299] Negotiation successful
0807 18:33:09.822984 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260807 18:33:09.823150 13143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:09.819390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.67:46029 (local address 127.0.0.1:57306)
0807 18:33:09.819670 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:09.819687 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:09.819806 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:09.820092 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:09.820095 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:09.820103 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 18:33:09.820217 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:09.820224 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:09.820938 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:09.820942 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:09.821946 (+ 1004us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:09.821959 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:09.822386 (+ 427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:09.822389 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:09.822486 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:09.822545 (+ 59us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 18:33:09.822828 (+ 283us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 18:33:09.822849 (+ 21us) client_negotiation.cc:769] Sending connection context
0807 18:33:09.822919 (+ 70us) client_negotiation.cc:240] Negotiation successful
0807 18:33:09.823007 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156}
I20260807 18:33:09.825933 12872 raft_consensus.cc:1261] T 72aeff6bf23645a88525429b8c145be1 P 34ede5db17834cfe86abe88e470bc41c [term 1 FOLLOWER]: Refusing update from remote peer a4ff3361f5784240bf29f7016bdaaa42: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:33:09.825989 12716 raft_consensus.cc:1261] T 72aeff6bf23645a88525429b8c145be1 P 62c0ffbc8d8d46fdb61ff8f094e1bb68 [term 1 FOLLOWER]: Refusing update from remote peer a4ff3361f5784240bf29f7016bdaaa42: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:33:09.826370 13111 consensus_queue.cc:1035] T 72aeff6bf23645a88525429b8c145be1 P a4ff3361f5784240bf29f7016bdaaa42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34ede5db17834cfe86abe88e470bc41c" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 44069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:33:09.826545 13108 consensus_queue.cc:1035] T 72aeff6bf23645a88525429b8c145be1 P a4ff3361f5784240bf29f7016bdaaa42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "62c0ffbc8d8d46fdb61ff8f094e1bb68" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 44813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:33:09.828161 13106 tablet_replica.cc:439] T 72aeff6bf23645a88525429b8c145be1 P 34ede5db17834cfe86abe88e470bc41c: TxnStatusTablet state changed. Reason: New leader a4ff3361f5784240bf29f7016bdaaa42. Latest consensus state: current_term: 1 leader_uuid: "a4ff3361f5784240bf29f7016bdaaa42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62c0ffbc8d8d46fdb61ff8f094e1bb68" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 44813 } } peers { permanent_uuid: "34ede5db17834cfe86abe88e470bc41c" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 44069 } } peers { permanent_uuid: "a4ff3361f5784240bf29f7016bdaaa42" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 46029 } } }
I20260807 18:33:09.828285 13106 tablet_replica.cc:442] T 72aeff6bf23645a88525429b8c145be1 P 34ede5db17834cfe86abe88e470bc41c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 18:33:09.829514 13108 tablet_replica.cc:439] T 72aeff6bf23645a88525429b8c145be1 P a4ff3361f5784240bf29f7016bdaaa42: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a4ff3361f5784240bf29f7016bdaaa42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62c0ffbc8d8d46fdb61ff8f094e1bb68" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 44813 } } peers { permanent_uuid: "34ede5db17834cfe86abe88e470bc41c" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 44069 } } peers { permanent_uuid: "a4ff3361f5784240bf29f7016bdaaa42" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 46029 } } }
I20260807 18:33:09.829633 13108 tablet_replica.cc:442] T 72aeff6bf23645a88525429b8c145be1 P a4ff3361f5784240bf29f7016bdaaa42: This TxnStatusTablet replica's current role is: LEADER
I20260807 18:33:09.829514 13114 mvcc.cc:205] Tried to move back new op lower bound from 7315978607922876416 to 7315978607827652608. Current Snapshot: MvccSnapshot[applied={T|T < 7315978607922876416}]
I20260807 18:33:09.830683 13106 tablet_replica.cc:439] T 72aeff6bf23645a88525429b8c145be1 P 34ede5db17834cfe86abe88e470bc41c: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a4ff3361f5784240bf29f7016bdaaa42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62c0ffbc8d8d46fdb61ff8f094e1bb68" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 44813 } } peers { permanent_uuid: "34ede5db17834cfe86abe88e470bc41c" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 44069 } } peers { permanent_uuid: "a4ff3361f5784240bf29f7016bdaaa42" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 46029 } } }
I20260807 18:33:09.830790 13106 tablet_replica.cc:442] T 72aeff6bf23645a88525429b8c145be1 P 34ede5db17834cfe86abe88e470bc41c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 18:33:09.830768 13109 tablet_replica.cc:439] T 72aeff6bf23645a88525429b8c145be1 P 62c0ffbc8d8d46fdb61ff8f094e1bb68: TxnStatusTablet state changed. Reason: New leader a4ff3361f5784240bf29f7016bdaaa42. Latest consensus state: current_term: 1 leader_uuid: "a4ff3361f5784240bf29f7016bdaaa42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62c0ffbc8d8d46fdb61ff8f094e1bb68" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 44813 } } peers { permanent_uuid: "34ede5db17834cfe86abe88e470bc41c" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 44069 } } peers { permanent_uuid: "a4ff3361f5784240bf29f7016bdaaa42" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 46029 } } }
I20260807 18:33:09.830885 13109 tablet_replica.cc:442] T 72aeff6bf23645a88525429b8c145be1 P 62c0ffbc8d8d46fdb61ff8f094e1bb68: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 18:33:09.831403 13115 mvcc.cc:205] Tried to move back new op lower bound from 7315978607922876416 to 7315978607827652608. Current Snapshot: MvccSnapshot[applied={T|T < 7315978607922876416 or (T in {7315978607922876416})}]
I20260807 18:33:09.832747 13155 tablet_replica.cc:439] T 72aeff6bf23645a88525429b8c145be1 P a4ff3361f5784240bf29f7016bdaaa42: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a4ff3361f5784240bf29f7016bdaaa42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62c0ffbc8d8d46fdb61ff8f094e1bb68" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 44813 } } peers { permanent_uuid: "34ede5db17834cfe86abe88e470bc41c" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 44069 } } peers { permanent_uuid: "a4ff3361f5784240bf29f7016bdaaa42" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 46029 } } }
I20260807 18:33:09.832906 13155 tablet_replica.cc:442] T 72aeff6bf23645a88525429b8c145be1 P a4ff3361f5784240bf29f7016bdaaa42: This TxnStatusTablet replica's current role is: LEADER
I20260807 18:33:09.834450 13116 mvcc.cc:205] Tried to move back new op lower bound from 7315978607922876416 to 7315978607827652608. Current Snapshot: MvccSnapshot[applied={T|T < 7315978607922876416}]
I20260807 18:33:09.834589 13109 tablet_replica.cc:439] T 72aeff6bf23645a88525429b8c145be1 P 62c0ffbc8d8d46fdb61ff8f094e1bb68: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a4ff3361f5784240bf29f7016bdaaa42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62c0ffbc8d8d46fdb61ff8f094e1bb68" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 44813 } } peers { permanent_uuid: "34ede5db17834cfe86abe88e470bc41c" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 44069 } } peers { permanent_uuid: "a4ff3361f5784240bf29f7016bdaaa42" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 46029 } } }
I20260807 18:33:09.834717 13109 tablet_replica.cc:442] T 72aeff6bf23645a88525429b8c145be1 P 62c0ffbc8d8d46fdb61ff8f094e1bb68: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 18:33:09.841814 13098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:09.837873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54882 (local address 127.11.14.66:44069)
0807 18:33:09.837993 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:09.837998 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:09.838013 (+ 15us) server_negotiation.cc:407] Connection header received
0807 18:33:09.838138 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:09.838141 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:09.838205 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:09.838316 (+ 111us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:33:09.838527 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:09.839313 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:09.840163 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:09.840509 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:09.840610 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:09.840849 (+ 239us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:33:09.841423 (+ 574us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:33:09.841543 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:09.841598 (+ 55us) server_negotiation.cc:299] Negotiation successful
0807 18:33:09.841655 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260807 18:33:09.853339 13096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:09.849755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.66:46099 (local address 127.11.14.67:46029)
0807 18:33:09.849900 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:09.849907 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:09.850152 (+ 245us) server_negotiation.cc:407] Connection header received
0807 18:33:09.850311 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:09.850315 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:09.850375 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:09.850453 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:33:09.850709 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:09.851221 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:09.852170 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:09.852404 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:09.852509 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:09.852846 (+ 337us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:33:09.852925 (+ 79us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:33:09.853002 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:09.853129 (+ 127us) server_negotiation.cc:299] Negotiation successful
0807 18:33:09.853183 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260807 18:33:09.853628 13158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:09.849660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.67:46029 (local address 127.11.14.66:46099)
0807 18:33:09.850065 (+ 405us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:09.850077 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:09.850205 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:09.850464 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:09.850467 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:09.850475 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 18:33:09.850605 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:09.850609 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:09.851336 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:09.851341 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:09.852032 (+ 691us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:09.852042 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:09.852548 (+ 506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:09.852555 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:09.852643 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:09.852687 (+ 44us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 18:33:09.853015 (+ 328us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 18:33:09.853024 (+ 9us) client_negotiation.cc:769] Sending connection context
0807 18:33:09.853103 (+ 79us) client_negotiation.cc:240] Negotiation successful
0807 18:33:09.853183 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":317,"thread_start_us":119,"threads_started":1}
I20260807 18:33:09.861004 13026 raft_consensus.cc:1261] T 9d65b116aa1e4027b215607ff2add626 P a4ff3361f5784240bf29f7016bdaaa42 [term 1 FOLLOWER]: Refusing update from remote peer 34ede5db17834cfe86abe88e470bc41c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:33:09.861471 12716 raft_consensus.cc:1261] T 9d65b116aa1e4027b215607ff2add626 P 62c0ffbc8d8d46fdb61ff8f094e1bb68 [term 1 FOLLOWER]: Refusing update from remote peer 34ede5db17834cfe86abe88e470bc41c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:33:09.861694 13106 consensus_queue.cc:1035] T 9d65b116aa1e4027b215607ff2add626 P 34ede5db17834cfe86abe88e470bc41c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4ff3361f5784240bf29f7016bdaaa42" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 46029 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:33:09.862069 13106 consensus_queue.cc:1035] T 9d65b116aa1e4027b215607ff2add626 P 34ede5db17834cfe86abe88e470bc41c [LEADER]: Connected to new peer: Peer: permanent_uuid: "62c0ffbc8d8d46fdb61ff8f094e1bb68" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 44813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:33:09.887956 13098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:09.883441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.67:57397 (local address 127.11.14.66:44069)
0807 18:33:09.883668 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:09.883673 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:09.883834 (+ 161us) server_negotiation.cc:407] Connection header received
0807 18:33:09.883997 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:09.884000 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:09.884049 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:09.884124 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:33:09.884649 (+ 525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:09.885442 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:09.886690 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:09.886897 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:09.886976 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:09.887433 (+ 457us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:33:09.887521 (+ 88us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:33:09.887646 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:09.887752 (+ 106us) server_negotiation.cc:299] Negotiation successful
0807 18:33:09.887804 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150}
I20260807 18:33:09.887975 13163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:09.883368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.66:44069 (local address 127.11.14.67:57397)
0807 18:33:09.883723 (+ 355us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:09.883738 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:09.883887 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:09.884280 (+ 393us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:09.884290 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:09.884298 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 18:33:09.884387 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:09.884391 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:09.885568 (+ 1177us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:09.885571 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:09.886559 (+ 988us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:09.886570 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:09.887010 (+ 440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:09.887014 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:09.887105 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:09.887161 (+ 56us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 18:33:09.887652 (+ 491us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 18:33:09.887660 (+ 8us) client_negotiation.cc:769] Sending connection context
0807 18:33:09.887721 (+ 61us) client_negotiation.cc:240] Negotiation successful
0807 18:33:09.887807 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":105,"threads_started":1}
I20260807 18:33:09.921532 12541 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53650:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:33:09.921736 12541 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53650:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:33:09.924515 12541 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a4c134b845384a56aa320c0f437b3512: Sending DeleteTablet for 3 replicas of tablet 9d65b116aa1e4027b215607ff2add626
I20260807 18:33:09.925297 13004 tablet_service.cc:1514] Processing DeleteTablet for tablet 9d65b116aa1e4027b215607ff2add626 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:33:09 UTC) from {username='kudu', principal='kudu/127.11.14.126@KRBTEST.COM'} at 127.0.0.1:57286
I20260807 18:33:09.925544 12694 tablet_service.cc:1514] Processing DeleteTablet for tablet 9d65b116aa1e4027b215607ff2add626 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:33:09 UTC) from {username='kudu', principal='kudu/127.11.14.126@KRBTEST.COM'} at 127.0.0.1:53726
I20260807 18:33:09.925806 13124 tablet_replica.cc:330] T 9d65b116aa1e4027b215607ff2add626 P a4ff3361f5784240bf29f7016bdaaa42: stopping tablet replica
I20260807 18:33:09.925818 13125 tablet_replica.cc:330] T 9d65b116aa1e4027b215607ff2add626 P 62c0ffbc8d8d46fdb61ff8f094e1bb68: stopping tablet replica
I20260807 18:33:09.925949 13124 raft_consensus.cc:2229] T 9d65b116aa1e4027b215607ff2add626 P a4ff3361f5784240bf29f7016bdaaa42 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:33:09.925961 13125 raft_consensus.cc:2229] T 9d65b116aa1e4027b215607ff2add626 P 62c0ffbc8d8d46fdb61ff8f094e1bb68 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:33:09.926088 13124 raft_consensus.cc:2258] T 9d65b116aa1e4027b215607ff2add626 P a4ff3361f5784240bf29f7016bdaaa42 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:33:09.926093 13125 raft_consensus.cc:2258] T 9d65b116aa1e4027b215607ff2add626 P 62c0ffbc8d8d46fdb61ff8f094e1bb68 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:33:09.927055 13124 ts_tablet_manager.cc:1905] T 9d65b116aa1e4027b215607ff2add626 P a4ff3361f5784240bf29f7016bdaaa42: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:33:09.927078 13125 ts_tablet_manager.cc:1905] T 9d65b116aa1e4027b215607ff2add626 P 62c0ffbc8d8d46fdb61ff8f094e1bb68: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:33:09.929746 12849 tablet_service.cc:1514] Processing DeleteTablet for tablet 9d65b116aa1e4027b215607ff2add626 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:33:09 UTC) from {username='kudu', principal='kudu/127.11.14.126@KRBTEST.COM'} at 127.0.0.1:54876
I20260807 18:33:09.929973 13126 tablet_replica.cc:330] T 9d65b116aa1e4027b215607ff2add626 P 34ede5db17834cfe86abe88e470bc41c: stopping tablet replica
I20260807 18:33:09.930112 13126 raft_consensus.cc:2229] T 9d65b116aa1e4027b215607ff2add626 P 34ede5db17834cfe86abe88e470bc41c [term 1 LEADER]: Raft consensus shutting down.
I20260807 18:33:09.930495 13124 ts_tablet_manager.cc:1918] T 9d65b116aa1e4027b215607ff2add626 P a4ff3361f5784240bf29f7016bdaaa42: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 18:33:09.930518 13126 raft_consensus.cc:2258] T 9d65b116aa1e4027b215607ff2add626 P 34ede5db17834cfe86abe88e470bc41c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:33:09.930569 13124 log.cc:1196] T 9d65b116aa1e4027b215607ff2add626 P a4ff3361f5784240bf29f7016bdaaa42: Deleting WAL directory at /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127579900523-11321-0/minicluster-data/ts-2/wal/wals/9d65b116aa1e4027b215607ff2add626
I20260807 18:33:09.930852 13124 ts_tablet_manager.cc:1939] T 9d65b116aa1e4027b215607ff2add626 P a4ff3361f5784240bf29f7016bdaaa42: Deleting consensus metadata
I20260807 18:33:09.931689 13126 ts_tablet_manager.cc:1905] T 9d65b116aa1e4027b215607ff2add626 P 34ede5db17834cfe86abe88e470bc41c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:33:09.931676 12527 catalog_manager.cc:4841] TS a4ff3361f5784240bf29f7016bdaaa42 (127.11.14.67:46029): tablet 9d65b116aa1e4027b215607ff2add626 (table test-table [id=b29e53041ae144b7a179ecc231ffa235]) successfully deleted
I20260807 18:33:09.935070 13126 ts_tablet_manager.cc:1918] T 9d65b116aa1e4027b215607ff2add626 P 34ede5db17834cfe86abe88e470bc41c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 18:33:09.935153 13126 log.cc:1196] T 9d65b116aa1e4027b215607ff2add626 P 34ede5db17834cfe86abe88e470bc41c: Deleting WAL directory at /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127579900523-11321-0/minicluster-data/ts-1/wal/wals/9d65b116aa1e4027b215607ff2add626
I20260807 18:33:09.935458 13126 ts_tablet_manager.cc:1939] T 9d65b116aa1e4027b215607ff2add626 P 34ede5db17834cfe86abe88e470bc41c: Deleting consensus metadata
I20260807 18:33:09.937057 13125 ts_tablet_manager.cc:1918] T 9d65b116aa1e4027b215607ff2add626 P 62c0ffbc8d8d46fdb61ff8f094e1bb68: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 18:33:09.937151 13125 log.cc:1196] T 9d65b116aa1e4027b215607ff2add626 P 62c0ffbc8d8d46fdb61ff8f094e1bb68: Deleting WAL directory at /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786127579900523-11321-0/minicluster-data/ts-0/wal/wals/9d65b116aa1e4027b215607ff2add626
I20260807 18:33:09.937407 13125 ts_tablet_manager.cc:1939] T 9d65b116aa1e4027b215607ff2add626 P 62c0ffbc8d8d46fdb61ff8f094e1bb68: Deleting consensus metadata
I20260807 18:33:09.937987 12525 catalog_manager.cc:4841] TS 34ede5db17834cfe86abe88e470bc41c (127.11.14.66:44069): tablet 9d65b116aa1e4027b215607ff2add626 (table test-table [id=b29e53041ae144b7a179ecc231ffa235]) successfully deleted
I20260807 18:33:09.938191 12525 catalog_manager.cc:4841] TS 62c0ffbc8d8d46fdb61ff8f094e1bb68 (127.11.14.65:44813): tablet 9d65b116aa1e4027b215607ff2add626 (table test-table [id=b29e53041ae144b7a179ecc231ffa235]) successfully deleted
Aug 07 18:33:09 dist-test-slave-6k22 krb5kdc[12492](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127589, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.11.14.65@KRBTEST.COM
I20260807 18:33:09.951784 13095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:09.938679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53734 (local address 127.11.14.65:44813)
0807 18:33:09.938826 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:09.938831 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:09.939143 (+ 312us) server_negotiation.cc:407] Connection header received
0807 18:33:09.939415 (+ 272us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:09.939418 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:09.939473 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:09.939579 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:33:09.940472 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:09.941273 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:09.942481 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:09.942876 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:09.942979 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:09.947041 (+ 4062us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:33:09.947059 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:33:09.947081 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:33:09.947119 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:33:09.949406 (+ 2287us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:09.949970 (+ 564us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:09.949977 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:09.949979 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:09.950177 (+ 198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:09.950490 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:09.950494 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:09.950495 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:09.950921 (+ 426us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:33:09.951140 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:09.951392 (+ 252us) server_negotiation.cc:299] Negotiation successful
0807 18:33:09.951525 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
W20260807 18:33:09.952442 12763 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53734
I20260807 18:33:09.964633 12597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:09.954505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53664 (local address 127.11.14.126:34397)
0807 18:33:09.954635 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:09.954640 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:09.955164 (+ 524us) server_negotiation.cc:407] Connection header received
0807 18:33:09.955277 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:09.955280 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:09.955341 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:09.955432 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:33:09.956403 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:09.957221 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:09.958396 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:09.958748 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:09.958869 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:09.960623 (+ 1754us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:33:09.960627 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:33:09.960630 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:33:09.960658 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:33:09.962418 (+ 1760us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:09.962929 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:09.962933 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:09.962935 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:09.963031 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:09.963343 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:09.963346 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:09.963348 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:09.963603 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:33:09.963756 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:09.964064 (+ 308us) server_negotiation.cc:299] Negotiation successful
0807 18:33:09.964201 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
W20260807 18:33:09.965092 12541 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53664
I20260807 18:33:09.967089 11321 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkz56XN/build/debug/bin/kudu with pid 12621
I20260807 18:33:09.976714 11321 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkz56XN/build/debug/bin/kudu with pid 12779
I20260807 18:33:09.985982 11321 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkz56XN/build/debug/bin/kudu with pid 12932
I20260807 18:33:09.995790 11321 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkz56XN/build/debug/bin/kudu with pid 12505
2026-08-07T18:33:10Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (3650 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127579900523-11321-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:33:10 dist-test-slave-6k22 krb5kdc[13174](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:33:10 dist-test-slave-6k22 krb5kdc[13174](info): set up 2 sockets
Aug 07 18:33:10 dist-test-slave-6k22 krb5kdc[13174](info): commencing operation
krb5kdc: starting...
W20260807 18:33:11.119710 11321 mini_kdc.cc:121] Time spent starting KDC: real 1.099s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127579900523-11321-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127579900523-11321-0/krb5kdc/test-user.keytab.
Aug 07 18:33:11 dist-test-slave-6k22 krb5kdc[13174](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127591, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:33:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:33:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.14.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.14.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.14.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.126.keytab.
Entry for principal kudu/127.11.14.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.126.keytab.
I20260807 18:33:11.254882 11321 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkz56XN/build/debug/bin/kudu
/tmp/dist-test-taskkz56XN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127579900523-11321-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127579900523-11321-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127579900523-11321-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127579900523-11321-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.14.126:36947
--webserver_interface=127.11.14.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.126.keytab
--principal=kudu/127.11.14.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.14.84:37377
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.14.126:36947
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127579900523-11321-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127579900523-11321-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127579900523-11321-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:33:11.372821 13189 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:33:11.373172 13189 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:33:11.373246 13189 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:33:11.378070 13189 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:33:11.378177 13189 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260807 18:33:11.378209 13189 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:33:11.378250 13189 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:33:11.378288 13189 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:33:11.378324 13189 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:33:11.384020 13189 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.14.84:37377
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127579900523-11321-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127579900523-11321-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.14.126:36947
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.126.keytab
--principal=kudu/127.11.14.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.14.126:36947
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127579900523-11321-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.14.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13189
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127579900523-11321-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:33:11.385245 13189 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:33:11.386364 13189 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:33:11.395236 13197 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:33:11.395241 13194 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:33:11.395470 13189 server_base.cc:1034] running on GCE node
W20260807 18:33:11.395350 13195 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:33:11.396157 13189 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:33:11.397280 13189 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:33:11.398464 13189 hybrid_clock.cc:648] HybridClock initialized: now 1786127591398443 us; error 32 us; skew 500 ppm
Aug 07 18:33:11 dist-test-slave-6k22 krb5kdc[13174](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127591, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:33:11.401463 13189 init.cc:377] Logged in from keytab as kudu/127.11.14.126@KRBTEST.COM (short username kudu)
I20260807 18:33:11.402493 13189 webserver.cc:460] Webserver started at http://127.11.14.126:33199/ using document root <none> and password file <none>
I20260807 18:33:11.403110 13189 fs_manager.cc:359] Metadata directory not provided
I20260807 18:33:11.403182 13189 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:33:11.403447 13189 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:33:11.405694 13189 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127579900523-11321-0/minicluster-data/master-0/data/instance:
uuid: "e09adbeddf294788bec26d0474bcf37c"
format_stamp: "Formatted at 2026-08-07 18:33:11 on dist-test-slave-6k22"
server_key: "41c5400c082302d19eacd8352070e511"
server_key_iv: "1b45dafb4cc1a2de2ebcc59f005a1c60"
server_key_version: "encryptionkey@0"
I20260807 18:33:11.406306 13189 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127579900523-11321-0/minicluster-data/master-0/wal/instance:
uuid: "e09adbeddf294788bec26d0474bcf37c"
format_stamp: "Formatted at 2026-08-07 18:33:11 on dist-test-slave-6k22"
server_key: "41c5400c082302d19eacd8352070e511"
server_key_iv: "1b45dafb4cc1a2de2ebcc59f005a1c60"
server_key_version: "encryptionkey@0"
I20260807 18:33:11.410440 13189 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260807 18:33:11.413113 13204 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:33:11.414433 13189 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 18:33:11.414587 13189 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127579900523-11321-0/minicluster-data/master-0/data,/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127579900523-11321-0/minicluster-data/master-0/wal
uuid: "e09adbeddf294788bec26d0474bcf37c"
format_stamp: "Formatted at 2026-08-07 18:33:11 on dist-test-slave-6k22"
server_key: "41c5400c082302d19eacd8352070e511"
server_key_iv: "1b45dafb4cc1a2de2ebcc59f005a1c60"
server_key_version: "encryptionkey@0"
I20260807 18:33:11.414713 13189 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127579900523-11321-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127579900523-11321-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127579900523-11321-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:33:11.436524 13189 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:33:11.439687 13189 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:33:11.439915 13189 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:33:11.448647 13189 rpc_server.cc:307] RPC server started. Bound to: 127.11.14.126:36947
I20260807 18:33:11.448671 13271 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.14.126:36947 every 8 connection(s)
I20260807 18:33:11.449681 13189 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127579900523-11321-0/minicluster-data/master-0/data/info.pb
I20260807 18:33:11.452282 11321 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkz56XN/build/debug/bin/kudu as pid 13189
I20260807 18:33:11.452383 11321 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127579900523-11321-0/minicluster-data/master-0/wal/instance
I20260807 18:33:11.452675 11321 external_mini_cluster.cc:1428] Setting key 6bef6a26220928fbb486f21f0a5acf3b
I20260807 18:33:11.452682 13272 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:33:11.460131 13272 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e09adbeddf294788bec26d0474bcf37c: Bootstrap starting.
Aug 07 18:33:11 dist-test-slave-6k22 krb5kdc[13174](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127591, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.14.126@KRBTEST.COM
I20260807 18:33:11.462793 13272 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e09adbeddf294788bec26d0474bcf37c: Neither blocks nor log segments found. Creating new log.
I20260807 18:33:11.463799 13272 log.cc:824] T 00000000000000000000000000000000 P e09adbeddf294788bec26d0474bcf37c: Log is configured to *not* fsync() on all Append() calls
I20260807 18:33:11.465989 13272 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e09adbeddf294788bec26d0474bcf37c: No bootstrap required, opened a new log
I20260807 18:33:11.468804 13277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:11.454177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54816 (local address 127.11.14.126:36947)
0807 18:33:11.454626 (+ 449us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:11.454647 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:11.454689 (+ 42us) server_negotiation.cc:407] Connection header received
0807 18:33:11.455537 (+ 848us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:11.455562 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:11.456014 (+ 452us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:11.456387 (+ 373us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:33:11.457033 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:11.458510 (+ 1477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:11.459347 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:11.460064 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:11.460235 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:11.463095 (+ 2860us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:33:11.463103 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:33:11.463120 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:33:11.463152 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:33:11.465818 (+ 2666us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:11.466386 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:11.466392 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:11.466396 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:11.466523 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:11.466835 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:11.466838 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:11.466842 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:11.467246 (+ 404us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:33:11.467461 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:11.467799 (+ 338us) server_negotiation.cc:299] Negotiation successful
0807 18:33:11.468087 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":296,"thread_start_us":124,"threads_started":1}
I20260807 18:33:11.469317 13272 raft_consensus.cc:348] T 00000000000000000000000000000000 P e09adbeddf294788bec26d0474bcf37c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e09adbeddf294788bec26d0474bcf37c" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 36947 } }
I20260807 18:33:11.469594 13272 raft_consensus.cc:374] T 00000000000000000000000000000000 P e09adbeddf294788bec26d0474bcf37c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:33:11.469653 13272 raft_consensus.cc:729] T 00000000000000000000000000000000 P e09adbeddf294788bec26d0474bcf37c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e09adbeddf294788bec26d0474bcf37c, State: Initialized, Role: FOLLOWER
I20260807 18:33:11.470124 13272 consensus_queue.cc:260] T 00000000000000000000000000000000 P e09adbeddf294788bec26d0474bcf37c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e09adbeddf294788bec26d0474bcf37c" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 36947 } }
I20260807 18:33:11.470252 13272 raft_consensus.cc:388] T 00000000000000000000000000000000 P e09adbeddf294788bec26d0474bcf37c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:33:11.470338 13272 raft_consensus.cc:482] T 00000000000000000000000000000000 P e09adbeddf294788bec26d0474bcf37c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:33:11.470446 13272 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e09adbeddf294788bec26d0474bcf37c [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:33:11.471594 13272 raft_consensus.cc:504] T 00000000000000000000000000000000 P e09adbeddf294788bec26d0474bcf37c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e09adbeddf294788bec26d0474bcf37c" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 36947 } }
I20260807 18:33:11.471954 13272 leader_election.cc:302] T 00000000000000000000000000000000 P e09adbeddf294788bec26d0474bcf37c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e09adbeddf294788bec26d0474bcf37c; no voters:
I20260807 18:33:11.472265 13272 leader_election.cc:288] T 00000000000000000000000000000000 P e09adbeddf294788bec26d0474bcf37c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:33:11.472390 13279 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e09adbeddf294788bec26d0474bcf37c [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:33:11.472632 13279 raft_consensus.cc:686] T 00000000000000000000000000000000 P e09adbeddf294788bec26d0474bcf37c [term 1 LEADER]: Becoming Leader. State: Replica: e09adbeddf294788bec26d0474bcf37c, State: Running, Role: LEADER
I20260807 18:33:11.472976 13279 consensus_queue.cc:237] T 00000000000000000000000000000000 P e09adbeddf294788bec26d0474bcf37c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e09adbeddf294788bec26d0474bcf37c" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 36947 } }
I20260807 18:33:11.473496 13272 sys_catalog.cc:564] T 00000000000000000000000000000000 P e09adbeddf294788bec26d0474bcf37c [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:33:11.474715 13281 sys_catalog.cc:455] T 00000000000000000000000000000000 P e09adbeddf294788bec26d0474bcf37c [sys.catalog]: SysCatalogTable state changed. Reason: New leader e09adbeddf294788bec26d0474bcf37c. Latest consensus state: current_term: 1 leader_uuid: "e09adbeddf294788bec26d0474bcf37c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e09adbeddf294788bec26d0474bcf37c" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 36947 } } }
I20260807 18:33:11.474861 13281 sys_catalog.cc:458] T 00000000000000000000000000000000 P e09adbeddf294788bec26d0474bcf37c [sys.catalog]: This master's current role is: LEADER
I20260807 18:33:11.475255 13280 sys_catalog.cc:455] T 00000000000000000000000000000000 P e09adbeddf294788bec26d0474bcf37c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e09adbeddf294788bec26d0474bcf37c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e09adbeddf294788bec26d0474bcf37c" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 36947 } } }
I20260807 18:33:11.475448 13280 sys_catalog.cc:458] T 00000000000000000000000000000000 P e09adbeddf294788bec26d0474bcf37c [sys.catalog]: This master's current role is: LEADER
I20260807 18:33:11.475622 13288 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:33:11.478394 13288 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:33:11.484622 13288 catalog_manager.cc:1347] Generated new cluster ID: e26add5d1dd04183a06667f3260195a6
I20260807 18:33:11.484715 13288 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:33:11.510347 13288 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:33:11.511626 13288 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:33:11.518285 13288 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e09adbeddf294788bec26d0474bcf37c: Generated new TSK 0
I20260807 18:33:11.519013 13288 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.14.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.14.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.14.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.65.keytab.
Entry for principal kudu/127.11.14.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.65.keytab.
I20260807 18:33:11.552799 11321 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkz56XN/build/debug/bin/kudu
/tmp/dist-test-taskkz56XN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127579900523-11321-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127579900523-11321-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127579900523-11321-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127579900523-11321-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.14.65:0
--local_ip_for_outbound_sockets=127.11.14.65
--webserver_interface=127.11.14.65
--webserver_port=0
--tserver_master_addrs=127.11.14.126:36947
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.65.keytab
--principal=kudu/127.11.14.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.14.84:37377
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127579900523-11321-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127579900523-11321-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127579900523-11321-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:33:11.671785 13300 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:33:11.672089 13300 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:33:11.672140 13300 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:33:11.676873 13300 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:33:11.676942 13300 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:33:11.677014 13300 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260807 18:33:11.677047 13300 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 18:33:11.677100 13300 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.14.65
I20260807 18:33:11.684732 13300 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.14.84:37377
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127579900523-11321-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127579900523-11321-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.65.keytab
--principal=kudu/127.11.14.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.14.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127579900523-11321-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.14.65
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.11.14.126:36947
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13300
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.14.65
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127579900523-11321-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:33:11.686426 13300 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:33:11.687635 13300 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:33:11.697887 13306 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:33:11.697909 13305 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:33:11.697923 13308 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:33:11.698856 13300 server_base.cc:1034] running on GCE node
I20260807 18:33:11.699436 13300 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:33:11.700157 13300 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:33:11.701361 13300 hybrid_clock.cc:648] HybridClock initialized: now 1786127591701349 us; error 32 us; skew 500 ppm
Aug 07 18:33:11 dist-test-slave-6k22 krb5kdc[13174](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127591, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:33:11.704249 13300 init.cc:377] Logged in from keytab as kudu/127.11.14.65@KRBTEST.COM (short username kudu)
I20260807 18:33:11.705226 13300 webserver.cc:460] Webserver started at http://127.11.14.65:38969/ using document root <none> and password file <none>
I20260807 18:33:11.705837 13300 fs_manager.cc:359] Metadata directory not provided
I20260807 18:33:11.705933 13300 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:33:11.706183 13300 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:33:11.708454 13300 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127579900523-11321-0/minicluster-data/ts-0/data/instance:
uuid: "97e3153b9b104e5a9fca75dcbb61571b"
format_stamp: "Formatted at 2026-08-07 18:33:11 on dist-test-slave-6k22"
server_key: "71d265fa724537f481e5b96a112ea27f"
server_key_iv: "0cf5979af083ff574f80a547533dd3a8"
server_key_version: "encryptionkey@0"
I20260807 18:33:11.709045 13300 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127579900523-11321-0/minicluster-data/ts-0/wal/instance:
uuid: "97e3153b9b104e5a9fca75dcbb61571b"
format_stamp: "Formatted at 2026-08-07 18:33:11 on dist-test-slave-6k22"
server_key: "71d265fa724537f481e5b96a112ea27f"
server_key_iv: "0cf5979af083ff574f80a547533dd3a8"
server_key_version: "encryptionkey@0"
I20260807 18:33:11.713042 13300 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 18:33:11.715659 13316 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:33:11.716957 13300 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 18:33:11.717105 13300 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127579900523-11321-0/minicluster-data/ts-0/data,/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127579900523-11321-0/minicluster-data/ts-0/wal
uuid: "97e3153b9b104e5a9fca75dcbb61571b"
format_stamp: "Formatted at 2026-08-07 18:33:11 on dist-test-slave-6k22"
server_key: "71d265fa724537f481e5b96a112ea27f"
server_key_iv: "0cf5979af083ff574f80a547533dd3a8"
server_key_version: "encryptionkey@0"
I20260807 18:33:11.717229 13300 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127579900523-11321-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127579900523-11321-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127579900523-11321-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:33:11.735126 13300 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:33:11.738363 13300 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:33:11.738586 13300 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:33:11.740310 13300 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:33:11.740386 13300 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:33:11.740473 13300 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:33:11.740514 13300 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 18:33:11 dist-test-slave-6k22 krb5kdc[13174](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127591, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.65@KRBTEST.COM for kudu/127.11.14.126@KRBTEST.COM
I20260807 18:33:11.752592 13300 rpc_server.cc:307] RPC server started. Bound to: 127.11.14.65:45267
I20260807 18:33:11.752677 13443 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.14.65:45267 every 8 connection(s)
I20260807 18:33:11.753856 13300 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127579900523-11321-0/minicluster-data/ts-0/data/info.pb
I20260807 18:33:11.756075 11321 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkz56XN/build/debug/bin/kudu as pid 13300
I20260807 18:33:11.756203 11321 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127579900523-11321-0/minicluster-data/ts-0/wal/instance
I20260807 18:33:11.756443 13277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:11.741095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.65:45041 (local address 127.11.14.126:36947)
0807 18:33:11.741240 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:11.741243 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:11.743106 (+ 1863us) server_negotiation.cc:407] Connection header received
0807 18:33:11.744367 (+ 1261us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:11.744370 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:11.744427 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:11.744535 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:33:11.745893 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:11.746484 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:11.748490 (+ 2006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:11.748846 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:11.748939 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:11.752238 (+ 3299us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:33:11.752242 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:33:11.752245 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:33:11.752275 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:33:11.754045 (+ 1770us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:11.754677 (+ 632us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:11.754681 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:11.754683 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:11.754784 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:11.755158 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:11.755161 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:11.755163 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:11.755363 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:33:11.755476 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:11.756129 (+ 653us) server_negotiation.cc:299] Negotiation successful
0807 18:33:11.756307 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260807 18:33:11.756527 11321 external_mini_cluster.cc:1428] Setting key 5bf84fd0586f1ddeabcf93403b048855
I20260807 18:33:11.757392 13326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:11.742296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:36947 (local address 127.11.14.65:45041)
0807 18:33:11.742941 (+ 645us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:11.742986 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:11.744145 (+ 1159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:11.744724 (+ 579us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:11.744738 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:11.745153 (+ 415us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:33:11.745718 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:11.745753 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:11.746684 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:11.746691 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:11.748341 (+ 1650us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:11.748355 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:11.749015 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:11.749021 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:11.749286 (+ 265us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:11.749955 (+ 669us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:33:11.749982 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:33:11.752031 (+ 2049us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:33:11.754158 (+ 2127us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:33:11.754163 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:33:11.754187 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:33:11.754529 (+ 342us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:33:11.754896 (+ 367us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:33:11.754899 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:33:11.754901 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:33:11.755043 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:33:11.755476 (+ 433us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:33:11.755484 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:33:11.755826 (+ 342us) client_negotiation.cc:769] Sending connection context
0807 18:33:11.756050 (+ 224us) client_negotiation.cc:240] Negotiation successful
0807 18:33:11.756420 (+ 370us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":348,"thread_start_us":180,"threads_started":1}
I20260807 18:33:11.758877 13444 heartbeater.cc:347] Connected to a master server at 127.11.14.126:36947
I20260807 18:33:11.759034 13444 heartbeater.cc:464] Registering TS with master...
I20260807 18:33:11.759663 13444 heartbeater.cc:511] Master 127.11.14.126:36947 requested a full tablet report, sending...
I20260807 18:33:11.761547 13220 ts_manager.cc:194] Registered new tserver with Master: 97e3153b9b104e5a9fca75dcbb61571b (127.11.14.65:45267)
I20260807 18:33:11.763326 13220 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.14.65@KRBTEST.COM'} at 127.11.14.65:45041
WARNING: no policy specified for kudu/127.11.14.66@KRBTEST.COM; defaulting to no policy
I20260807 18:33:11.770886 13277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:11.761207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.65:42777 (local address 127.11.14.126:36947)
0807 18:33:11.761337 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:11.761341 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:11.763827 (+ 2486us) server_negotiation.cc:407] Connection header received
0807 18:33:11.763984 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:11.763986 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:11.764049 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:11.764123 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:33:11.764719 (+ 596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:11.765238 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:11.766077 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:11.766306 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:11.766412 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:11.767700 (+ 1288us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:33:11.767708 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:33:11.767710 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:33:11.767734 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:33:11.769237 (+ 1503us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:11.769662 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:11.769666 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:11.769667 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:11.769750 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:11.770041 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:11.770046 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:11.770047 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:11.770222 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:33:11.770342 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:11.770599 (+ 257us) server_negotiation.cc:299] Negotiation successful
0807 18:33:11.770728 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260807 18:33:11.770895 13451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:11.761163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:36947 (local address 127.11.14.65:42777)
0807 18:33:11.763736 (+ 2573us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:11.763749 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:11.763875 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:11.764239 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:11.764243 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:11.764486 (+ 243us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:33:11.764600 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:11.764606 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:11.765372 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:11.765380 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:11.765944 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:11.765952 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:11.766429 (+ 477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:11.766433 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:11.766532 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:11.767168 (+ 636us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:33:11.767180 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:33:11.767515 (+ 335us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:33:11.769355 (+ 1840us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:33:11.769358 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:33:11.769360 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:33:11.769561 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:33:11.769845 (+ 284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:33:11.769849 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:33:11.769851 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:33:11.769940 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:33:11.770332 (+ 392us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:33:11.770335 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:33:11.770434 (+ 99us) client_negotiation.cc:769] Sending connection context
0807 18:33:11.770543 (+ 109us) client_negotiation.cc:240] Negotiation successful
0807 18:33:11.770686 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2489,"thread_start_us":88,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.14.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.14.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.66.keytab.
Entry for principal kudu/127.11.14.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.66.keytab.
I20260807 18:33:11.790726 11321 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkz56XN/build/debug/bin/kudu
/tmp/dist-test-taskkz56XN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127579900523-11321-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127579900523-11321-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127579900523-11321-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127579900523-11321-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.14.66:0
--local_ip_for_outbound_sockets=127.11.14.66
--webserver_interface=127.11.14.66
--webserver_port=0
--tserver_master_addrs=127.11.14.126:36947
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.66.keytab
--principal=kudu/127.11.14.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.14.84:37377
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127579900523-11321-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127579900523-11321-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127579900523-11321-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:33:11.910678 13453 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:33:11.911000 13453 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:33:11.911082 13453 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:33:11.915158 13453 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:33:11.915266 13453 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:33:11.915337 13453 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260807 18:33:11.915382 13453 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 18:33:11.915457 13453 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.14.66
I20260807 18:33:11.921150 13453 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.14.84:37377
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127579900523-11321-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127579900523-11321-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.66.keytab
--principal=kudu/127.11.14.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.14.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127579900523-11321-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.14.66
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.11.14.126:36947
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13453
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.14.66
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127579900523-11321-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:33:11.922370 13453 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:33:11.923448 13453 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:33:11.934010 13461 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:33:11.933990 13459 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:33:11.933990 13458 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:33:11.934461 13453 server_base.cc:1034] running on GCE node
I20260807 18:33:11.934864 13453 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:33:11.935505 13453 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:33:11.936721 13453 hybrid_clock.cc:648] HybridClock initialized: now 1786127591936686 us; error 46 us; skew 500 ppm
Aug 07 18:33:11 dist-test-slave-6k22 krb5kdc[13174](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127591, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:33:11.939524 13453 init.cc:377] Logged in from keytab as kudu/127.11.14.66@KRBTEST.COM (short username kudu)
I20260807 18:33:11.940454 13453 webserver.cc:460] Webserver started at http://127.11.14.66:35609/ using document root <none> and password file <none>
I20260807 18:33:11.941027 13453 fs_manager.cc:359] Metadata directory not provided
I20260807 18:33:11.941093 13453 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:33:11.941267 13453 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:33:11.943445 13453 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127579900523-11321-0/minicluster-data/ts-1/data/instance:
uuid: "99ba8d46c261419aa0813e9e10dc6cd1"
format_stamp: "Formatted at 2026-08-07 18:33:11 on dist-test-slave-6k22"
server_key: "17f4380269a52ff716ab067fe6f3b87d"
server_key_iv: "3296f119c34a76a8c1be67af7a04b3b5"
server_key_version: "encryptionkey@0"
I20260807 18:33:11.944041 13453 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127579900523-11321-0/minicluster-data/ts-1/wal/instance:
uuid: "99ba8d46c261419aa0813e9e10dc6cd1"
format_stamp: "Formatted at 2026-08-07 18:33:11 on dist-test-slave-6k22"
server_key: "17f4380269a52ff716ab067fe6f3b87d"
server_key_iv: "3296f119c34a76a8c1be67af7a04b3b5"
server_key_version: "encryptionkey@0"
I20260807 18:33:11.947955 13453 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 18:33:11.950547 13468 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:33:11.952080 13453 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:33:11.952183 13453 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127579900523-11321-0/minicluster-data/ts-1/data,/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127579900523-11321-0/minicluster-data/ts-1/wal
uuid: "99ba8d46c261419aa0813e9e10dc6cd1"
format_stamp: "Formatted at 2026-08-07 18:33:11 on dist-test-slave-6k22"
server_key: "17f4380269a52ff716ab067fe6f3b87d"
server_key_iv: "3296f119c34a76a8c1be67af7a04b3b5"
server_key_version: "encryptionkey@0"
I20260807 18:33:11.952281 13453 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127579900523-11321-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127579900523-11321-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127579900523-11321-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:33:11.993983 13453 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:33:11.997112 13453 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:33:11.997287 13453 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:33:11.998907 13453 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:33:11.998961 13453 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:33:11.999006 13453 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:33:11.999059 13453 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:33:12.010370 13453 rpc_server.cc:307] RPC server started. Bound to: 127.11.14.66:45171
Aug 07 18:33:12 dist-test-slave-6k22 krb5kdc[13174](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127591, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.66@KRBTEST.COM for kudu/127.11.14.126@KRBTEST.COM
I20260807 18:33:12.010411 13590 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.14.66:45171 every 8 connection(s)
I20260807 18:33:12.011490 13453 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127579900523-11321-0/minicluster-data/ts-1/data/info.pb
I20260807 18:33:12.014837 13277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:11.999791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.66:57971 (local address 127.11.14.126:36947)
0807 18:33:11.999934 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:11.999937 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:12.001103 (+ 1166us) server_negotiation.cc:407] Connection header received
0807 18:33:12.002277 (+ 1174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:12.002283 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:12.002352 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:12.002419 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:33:12.004476 (+ 2057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:12.005370 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:12.007361 (+ 1991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:12.007639 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:12.007745 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:12.011239 (+ 3494us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:33:12.011242 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:33:12.011244 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:33:12.011273 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:33:12.013017 (+ 1744us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:12.013513 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:12.013516 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:12.013517 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:12.013594 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:12.013898 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:12.013901 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:12.013902 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:12.014079 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:33:12.014155 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:12.014608 (+ 453us) server_negotiation.cc:299] Negotiation successful
0807 18:33:12.014709 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260807 18:33:12.015799 13478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:12.000418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:36947 (local address 127.11.14.66:57971)
0807 18:33:12.000943 (+ 525us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:12.000988 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:12.001996 (+ 1008us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:12.002955 (+ 959us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:12.002967 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:12.003430 (+ 463us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:33:12.004270 (+ 840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:12.004295 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:12.005540 (+ 1245us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:12.005546 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:12.007206 (+ 1660us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:12.007217 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:12.007827 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:12.007831 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:12.008092 (+ 261us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:12.008980 (+ 888us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:33:12.009006 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:33:12.011013 (+ 2007us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:33:12.013134 (+ 2121us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:33:12.013139 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:33:12.013152 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:33:12.013422 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:33:12.013675 (+ 253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:33:12.013677 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:33:12.013679 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:33:12.013809 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:33:12.014167 (+ 358us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:33:12.014172 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:33:12.014402 (+ 230us) client_negotiation.cc:769] Sending connection context
0807 18:33:12.014561 (+ 159us) client_negotiation.cc:240] Negotiation successful
0807 18:33:12.014815 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":300,"thread_start_us":131,"threads_started":1}
I20260807 18:33:12.017076 13591 heartbeater.cc:347] Connected to a master server at 127.11.14.126:36947
I20260807 18:33:12.017221 13591 heartbeater.cc:464] Registering TS with master...
I20260807 18:33:12.017839 13591 heartbeater.cc:511] Master 127.11.14.126:36947 requested a full tablet report, sending...
I20260807 18:33:12.018337 11321 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkz56XN/build/debug/bin/kudu as pid 13453
I20260807 18:33:12.018479 11321 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127579900523-11321-0/minicluster-data/ts-1/wal/instance
I20260807 18:33:12.018936 11321 external_mini_cluster.cc:1428] Setting key 3dde1228438f05dd3c812c55ccd99257
I20260807 18:33:12.019066 13221 ts_manager.cc:194] Registered new tserver with Master: 99ba8d46c261419aa0813e9e10dc6cd1 (127.11.14.66:45171)
I20260807 18:33:12.019918 13221 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.14.66@KRBTEST.COM'} at 127.11.14.66:57971
I20260807 18:33:12.028494 13597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:12.019703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:36947 (local address 127.11.14.66:58145)
0807 18:33:12.019998 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:12.020014 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:12.020101 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:12.020748 (+ 647us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:12.020752 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:12.021003 (+ 251us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:33:12.021161 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:12.021173 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:12.022008 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:12.022014 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:12.022949 (+ 935us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:12.022960 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:12.023618 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:12.023621 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:12.023710 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:12.024305 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:33:12.024319 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:33:12.024749 (+ 430us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:33:12.026883 (+ 2134us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:33:12.026886 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:33:12.026887 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:33:12.027085 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:33:12.027403 (+ 318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:33:12.027405 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:33:12.027406 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:33:12.027482 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:33:12.028061 (+ 579us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:33:12.028063 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:33:12.028142 (+ 79us) client_negotiation.cc:769] Sending connection context
0807 18:33:12.028228 (+ 86us) client_negotiation.cc:240] Negotiation successful
0807 18:33:12.028336 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":94,"threads_started":1}
I20260807 18:33:12.028632 13277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:12.020353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.66:58145 (local address 127.11.14.126:36947)
0807 18:33:12.020460 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:12.020465 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:12.020480 (+ 15us) server_negotiation.cc:407] Connection header received
0807 18:33:12.020515 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:12.020520 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:12.020560 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:12.020667 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:33:12.021284 (+ 617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:12.021873 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:12.023084 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:12.023461 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:12.023619 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:12.024903 (+ 1284us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:33:12.024910 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:33:12.024916 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:33:12.024948 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:33:12.026747 (+ 1799us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:12.027190 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:12.027195 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:12.027199 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:12.027299 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:12.027607 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:12.027613 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:12.027617 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:12.027956 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:33:12.028077 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:12.028319 (+ 242us) server_negotiation.cc:299] Negotiation successful
0807 18:33:12.028457 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
WARNING: no policy specified for kudu/127.11.14.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.14.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.14.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.67.keytab.
Entry for principal kudu/127.11.14.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.67.keytab.
I20260807 18:33:12.048902 11321 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkz56XN/build/debug/bin/kudu
/tmp/dist-test-taskkz56XN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127579900523-11321-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127579900523-11321-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127579900523-11321-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127579900523-11321-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.14.67:0
--local_ip_for_outbound_sockets=127.11.14.67
--webserver_interface=127.11.14.67
--webserver_port=0
--tserver_master_addrs=127.11.14.126:36947
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.67.keytab
--principal=kudu/127.11.14.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.14.84:37377
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127579900523-11321-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127579900523-11321-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127579900523-11321-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:33:12.171770 13600 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:33:12.172043 13600 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:33:12.172091 13600 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:33:12.178059 13600 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:33:12.178149 13600 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:33:12.178206 13600 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260807 18:33:12.178227 13600 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 18:33:12.178303 13600 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.14.67
I20260807 18:33:12.183995 13600 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.14.84:37377
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127579900523-11321-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127579900523-11321-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.67.keytab
--principal=kudu/127.11.14.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.14.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127579900523-11321-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.14.67
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.11.14.126:36947
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13600
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.14.67
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127579900523-11321-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:33:12.185074 13600 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:33:12.186128 13600 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:33:12.195950 13606 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:33:12.196018 13608 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:33:12.195981 13605 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:33:12.196280 13600 server_base.cc:1034] running on GCE node
I20260807 18:33:12.196693 13600 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:33:12.197350 13600 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:33:12.198540 13600 hybrid_clock.cc:648] HybridClock initialized: now 1786127592198515 us; error 34 us; skew 500 ppm
Aug 07 18:33:12 dist-test-slave-6k22 krb5kdc[13174](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127592, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:33:12.201371 13600 init.cc:377] Logged in from keytab as kudu/127.11.14.67@KRBTEST.COM (short username kudu)
I20260807 18:33:12.202261 13600 webserver.cc:460] Webserver started at http://127.11.14.67:38859/ using document root <none> and password file <none>
I20260807 18:33:12.202785 13600 fs_manager.cc:359] Metadata directory not provided
I20260807 18:33:12.202843 13600 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:33:12.203012 13600 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:33:12.205191 13600 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127579900523-11321-0/minicluster-data/ts-2/data/instance:
uuid: "3596f9392f8746a09722d927914e3f20"
format_stamp: "Formatted at 2026-08-07 18:33:12 on dist-test-slave-6k22"
server_key: "f15909ba200241c2bb67c5a53bcfb68c"
server_key_iv: "5d1dd9da703cae4c2b4581a2ed9ff1e7"
server_key_version: "encryptionkey@0"
I20260807 18:33:12.205713 13600 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127579900523-11321-0/minicluster-data/ts-2/wal/instance:
uuid: "3596f9392f8746a09722d927914e3f20"
format_stamp: "Formatted at 2026-08-07 18:33:12 on dist-test-slave-6k22"
server_key: "f15909ba200241c2bb67c5a53bcfb68c"
server_key_iv: "5d1dd9da703cae4c2b4581a2ed9ff1e7"
server_key_version: "encryptionkey@0"
I20260807 18:33:12.209583 13600 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 18:33:12.212013 13615 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:33:12.213267 13600 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 18:33:12.213403 13600 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127579900523-11321-0/minicluster-data/ts-2/data,/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127579900523-11321-0/minicluster-data/ts-2/wal
uuid: "3596f9392f8746a09722d927914e3f20"
format_stamp: "Formatted at 2026-08-07 18:33:12 on dist-test-slave-6k22"
server_key: "f15909ba200241c2bb67c5a53bcfb68c"
server_key_iv: "5d1dd9da703cae4c2b4581a2ed9ff1e7"
server_key_version: "encryptionkey@0"
I20260807 18:33:12.213518 13600 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127579900523-11321-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127579900523-11321-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127579900523-11321-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:33:12.244900 13600 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:33:12.248071 13600 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:33:12.248301 13600 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:33:12.250329 13600 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:33:12.250401 13600 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:33:12.250470 13600 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:33:12.250500 13600 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 18:33:12 dist-test-slave-6k22 krb5kdc[13174](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127592, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.67@KRBTEST.COM for kudu/127.11.14.126@KRBTEST.COM
I20260807 18:33:12.261868 13600 rpc_server.cc:307] RPC server started. Bound to: 127.11.14.67:41295
I20260807 18:33:12.261896 13740 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.14.67:41295 every 8 connection(s)
I20260807 18:33:12.262766 13600 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127579900523-11321-0/minicluster-data/ts-2/data/info.pb
I20260807 18:33:12.265367 13277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:12.251170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.67:42793 (local address 127.11.14.126:36947)
0807 18:33:12.251327 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:12.251332 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:12.252329 (+ 997us) server_negotiation.cc:407] Connection header received
0807 18:33:12.253411 (+ 1082us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:12.253416 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:12.253483 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:12.253542 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:33:12.255500 (+ 1958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:12.256478 (+ 978us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:12.257840 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:12.258232 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:12.258378 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:12.261719 (+ 3341us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:33:12.261722 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:33:12.261724 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:33:12.261751 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:33:12.263481 (+ 1730us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:12.264052 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:12.264055 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:12.264056 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:12.264139 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:12.264410 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:12.264413 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:12.264413 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:12.264592 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:33:12.264669 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:12.265140 (+ 471us) server_negotiation.cc:299] Negotiation successful
0807 18:33:12.265241 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260807 18:33:12.266206 13626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:12.251673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:36947 (local address 127.11.14.67:42793)
0807 18:33:12.252147 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:12.252201 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:12.253083 (+ 882us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:12.254052 (+ 969us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:12.254064 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:12.254550 (+ 486us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:33:12.255315 (+ 765us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:12.255352 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:12.256625 (+ 1273us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:12.256630 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:12.257732 (+ 1102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:12.257742 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:12.258449 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:12.258455 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:12.258691 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:12.259392 (+ 701us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:33:12.259418 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:33:12.261545 (+ 2127us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:33:12.263636 (+ 2091us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:33:12.263642 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:33:12.263655 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:33:12.263953 (+ 298us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:33:12.264206 (+ 253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:33:12.264209 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:33:12.264210 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:33:12.264334 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:33:12.264698 (+ 364us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:33:12.264704 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:33:12.264935 (+ 231us) client_negotiation.cc:769] Sending connection context
0807 18:33:12.265090 (+ 155us) client_negotiation.cc:240] Negotiation successful
0807 18:33:12.265352 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":128,"threads_started":1}
I20260807 18:33:12.267102 11321 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkz56XN/build/debug/bin/kudu as pid 13600
I20260807 18:33:12.267234 11321 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127579900523-11321-0/minicluster-data/ts-2/wal/instance
I20260807 18:33:12.267524 13741 heartbeater.cc:347] Connected to a master server at 127.11.14.126:36947
I20260807 18:33:12.267699 13741 heartbeater.cc:464] Registering TS with master...
I20260807 18:33:12.267699 11321 external_mini_cluster.cc:1428] Setting key db7323900a286be8914def8f11e59ca6
I20260807 18:33:12.268280 13741 heartbeater.cc:511] Master 127.11.14.126:36947 requested a full tablet report, sending...
I20260807 18:33:12.269483 13220 ts_manager.cc:194] Registered new tserver with Master: 3596f9392f8746a09722d927914e3f20 (127.11.14.67:41295)
I20260807 18:33:12.270278 13220 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.14.67@KRBTEST.COM'} at 127.11.14.67:42793
I20260807 18:33:12.276904 13748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:12.270043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:36947 (local address 127.11.14.67:41457)
0807 18:33:12.270316 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:12.270330 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:12.270500 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:12.270751 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:12.270754 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:12.270965 (+ 211us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:33:12.271061 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:12.271067 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:12.271981 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:12.271983 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:12.272537 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:12.272544 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:12.272955 (+ 411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:12.272958 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:12.273028 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:12.273562 (+ 534us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:33:12.273574 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:33:12.273899 (+ 325us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:33:12.275611 (+ 1712us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:33:12.275614 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:33:12.275643 (+ 29us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:33:12.275860 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:33:12.276107 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:33:12.276109 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:33:12.276110 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:33:12.276185 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:33:12.276496 (+ 311us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:33:12.276498 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:33:12.276573 (+ 75us) client_negotiation.cc:769] Sending connection context
0807 18:33:12.276652 (+ 79us) client_negotiation.cc:240] Negotiation successful
0807 18:33:12.276755 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":107,"threads_started":1}
I20260807 18:33:12.276911 13277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:12.270084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.67:41457 (local address 127.11.14.126:36947)
0807 18:33:12.270203 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:12.270207 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:12.270425 (+ 218us) server_negotiation.cc:407] Connection header received
0807 18:33:12.270603 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:12.270606 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:12.270654 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:12.270734 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:33:12.271219 (+ 485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:12.271844 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:12.272642 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:12.272850 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:12.272934 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:12.274015 (+ 1081us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:33:12.274019 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:33:12.274038 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:33:12.274066 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:33:12.275450 (+ 1384us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:12.275949 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:12.275952 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:12.275953 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:12.276027 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:12.276264 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:12.276266 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:12.276267 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:12.276420 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:33:12.276495 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:12.276712 (+ 217us) server_negotiation.cc:299] Negotiation successful
0807 18:33:12.276807 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260807 18:33:12.281695 11321 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 18:33:12 dist-test-slave-6k22 krb5kdc[13174](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127592, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 07 18:33:12 dist-test-slave-6k22 krb5kdc[13174](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127592, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.14.126@KRBTEST.COM
I20260807 18:33:12.309523 13277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:12.301316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54826 (local address 127.11.14.126:36947)
0807 18:33:12.301463 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:12.301466 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:12.301479 (+ 13us) server_negotiation.cc:407] Connection header received
0807 18:33:12.301523 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:12.301525 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:12.301564 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:12.301631 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:33:12.302389 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:12.302921 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:12.303835 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:12.304054 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:12.304139 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:12.306649 (+ 2510us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:33:12.306653 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:33:12.306655 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:33:12.306682 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:33:12.308013 (+ 1331us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:12.308399 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:12.308402 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:12.308403 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:12.308477 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:12.308709 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:12.308711 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:12.308712 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:12.308913 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:33:12.309016 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:12.309219 (+ 203us) server_negotiation.cc:299] Negotiation successful
0807 18:33:12.309319 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260807 18:33:12.312651 13220 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54826:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 18:33:12.316409 13220 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 18:33:12.332135 13761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:12.325867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36712 (local address 127.11.14.65:45267)
0807 18:33:12.327805 (+ 1938us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:12.327811 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:12.327833 (+ 22us) server_negotiation.cc:407] Connection header received
0807 18:33:12.327903 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:12.327907 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:12.328100 (+ 193us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:12.328259 (+ 159us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:33:12.328483 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:12.329385 (+ 902us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:12.330815 (+ 1430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:12.331414 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:12.331521 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:12.331714 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:12.331894 (+ 180us) server_negotiation.cc:299] Negotiation successful
0807 18:33:12.331992 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1812,"thread_start_us":92,"threads_started":1}
I20260807 18:33:12.332135 13760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:12.325700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.65:45267 (local address 127.0.0.1:36712)
0807 18:33:12.326545 (+ 845us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:12.326598 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:12.326735 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:12.328243 (+ 1508us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:12.328246 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:12.328265 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:33:12.328354 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:12.328363 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:12.329520 (+ 1157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:12.329525 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:12.330684 (+ 1159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:12.330694 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:12.331536 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:12.331539 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:12.331688 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:12.331731 (+ 43us) client_negotiation.cc:769] Sending connection context
0807 18:33:12.331859 (+ 128us) client_negotiation.cc:240] Negotiation successful
0807 18:33:12.331975 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":665,"mutex_wait_us":48,"thread_start_us":89,"threads_started":1}
I20260807 18:33:12.333109 13763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:12.325869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.67:41295 (local address 127.0.0.1:50110)
0807 18:33:12.327238 (+ 1369us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:12.327249 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:12.327329 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:12.327817 (+ 488us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:12.327824 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:12.327851 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:33:12.327986 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:12.327992 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:12.329510 (+ 1518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:12.329513 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:12.331208 (+ 1695us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:12.331220 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:12.332726 (+ 1506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:12.332730 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:12.332840 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:12.332863 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 18:33:12.332912 (+ 49us) client_negotiation.cc:240] Negotiation successful
0807 18:33:12.332974 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1321,"mutex_wait_us":200,"thread_start_us":108,"threads_started":1}
I20260807 18:33:12.333706 13764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:12.326543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50110 (local address 127.11.14.67:41295)
0807 18:33:12.327045 (+ 502us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:12.327054 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:12.327332 (+ 278us) server_negotiation.cc:407] Connection header received
0807 18:33:12.327405 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:12.327409 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:12.327647 (+ 238us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:12.327807 (+ 160us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:33:12.328116 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:12.329376 (+ 1260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:12.331344 (+ 1968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:12.332593 (+ 1249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:12.333334 (+ 741us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:12.333488 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:12.333529 (+ 41us) server_negotiation.cc:299] Negotiation successful
0807 18:33:12.333595 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":412,"thread_start_us":116,"threads_started":1}
I20260807 18:33:12.335125 13371 tablet_service.cc:1467] Processing CreateTablet for tablet c36b772e33454c9aaf039ceebfc1177c (DEFAULT_TABLE table=test-table [id=f3b352ba6e6947c18b4185308dae76bc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:33:12.336181 13371 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c36b772e33454c9aaf039ceebfc1177c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:33:12.335896 13673 tablet_service.cc:1467] Processing CreateTablet for tablet c36b772e33454c9aaf039ceebfc1177c (DEFAULT_TABLE table=test-table [id=f3b352ba6e6947c18b4185308dae76bc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:33:12.336839 13673 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c36b772e33454c9aaf039ceebfc1177c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:33:12.341287 13762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:12.326193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.66:45171 (local address 127.0.0.1:55458)
0807 18:33:12.327449 (+ 1256us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:12.327458 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:12.327517 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:12.328126 (+ 609us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:12.328130 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:12.328149 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:33:12.328260 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:12.328266 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:12.329677 (+ 1411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:12.329680 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:12.331208 (+ 1528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:12.331220 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:12.332614 (+ 1394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:12.332618 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:12.332720 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:12.332966 (+ 246us) client_negotiation.cc:769] Sending connection context
0807 18:33:12.333034 (+ 68us) client_negotiation.cc:240] Negotiation successful
0807 18:33:12.341142 (+ 8108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1218,"mutex_wait_us":374,"thread_start_us":82,"threads_started":1}
I20260807 18:33:12.342048 13765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:12.326627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55458 (local address 127.11.14.66:45171)
0807 18:33:12.327132 (+ 505us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:12.327141 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:12.327541 (+ 400us) server_negotiation.cc:407] Connection header received
0807 18:33:12.327647 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:12.327652 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:12.327833 (+ 181us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:12.327969 (+ 136us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:33:12.328410 (+ 441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:12.329557 (+ 1147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:12.331370 (+ 1813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:12.332188 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:12.332298 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:12.332913 (+ 615us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:12.341767 (+ 8854us) server_negotiation.cc:299] Negotiation successful
0807 18:33:12.341901 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":412,"thread_start_us":113,"threads_started":1}
I20260807 18:33:12.342603 13766 tablet_bootstrap.cc:492] T c36b772e33454c9aaf039ceebfc1177c P 97e3153b9b104e5a9fca75dcbb61571b: Bootstrap starting.
I20260807 18:33:12.344802 13766 tablet_bootstrap.cc:654] T c36b772e33454c9aaf039ceebfc1177c P 97e3153b9b104e5a9fca75dcbb61571b: Neither blocks nor log segments found. Creating new log.
I20260807 18:33:12.344615 13515 tablet_service.cc:1467] Processing CreateTablet for tablet c36b772e33454c9aaf039ceebfc1177c (DEFAULT_TABLE table=test-table [id=f3b352ba6e6947c18b4185308dae76bc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:33:12.345556 13515 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c36b772e33454c9aaf039ceebfc1177c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:33:12.345587 13766 log.cc:824] T c36b772e33454c9aaf039ceebfc1177c P 97e3153b9b104e5a9fca75dcbb61571b: Log is configured to *not* fsync() on all Append() calls
I20260807 18:33:12.346290 13767 tablet_bootstrap.cc:492] T c36b772e33454c9aaf039ceebfc1177c P 3596f9392f8746a09722d927914e3f20: Bootstrap starting.
I20260807 18:33:12.347767 13766 tablet_bootstrap.cc:492] T c36b772e33454c9aaf039ceebfc1177c P 97e3153b9b104e5a9fca75dcbb61571b: No bootstrap required, opened a new log
I20260807 18:33:12.347966 13766 ts_tablet_manager.cc:1397] T c36b772e33454c9aaf039ceebfc1177c P 97e3153b9b104e5a9fca75dcbb61571b: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.003s
I20260807 18:33:12.348531 13767 tablet_bootstrap.cc:654] T c36b772e33454c9aaf039ceebfc1177c P 3596f9392f8746a09722d927914e3f20: Neither blocks nor log segments found. Creating new log.
I20260807 18:33:12.349450 13767 log.cc:824] T c36b772e33454c9aaf039ceebfc1177c P 3596f9392f8746a09722d927914e3f20: Log is configured to *not* fsync() on all Append() calls
I20260807 18:33:12.351649 13767 tablet_bootstrap.cc:492] T c36b772e33454c9aaf039ceebfc1177c P 3596f9392f8746a09722d927914e3f20: No bootstrap required, opened a new log
I20260807 18:33:12.351862 13767 ts_tablet_manager.cc:1397] T c36b772e33454c9aaf039ceebfc1177c P 3596f9392f8746a09722d927914e3f20: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260807 18:33:12.352620 13771 tablet_bootstrap.cc:492] T c36b772e33454c9aaf039ceebfc1177c P 99ba8d46c261419aa0813e9e10dc6cd1: Bootstrap starting.
I20260807 18:33:12.352502 13766 raft_consensus.cc:348] T c36b772e33454c9aaf039ceebfc1177c P 97e3153b9b104e5a9fca75dcbb61571b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97e3153b9b104e5a9fca75dcbb61571b" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 45267 } } peers { permanent_uuid: "3596f9392f8746a09722d927914e3f20" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 41295 } } peers { permanent_uuid: "99ba8d46c261419aa0813e9e10dc6cd1" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 45171 } }
I20260807 18:33:12.352794 13766 raft_consensus.cc:374] T c36b772e33454c9aaf039ceebfc1177c P 97e3153b9b104e5a9fca75dcbb61571b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:33:12.352867 13766 raft_consensus.cc:729] T c36b772e33454c9aaf039ceebfc1177c P 97e3153b9b104e5a9fca75dcbb61571b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97e3153b9b104e5a9fca75dcbb61571b, State: Initialized, Role: FOLLOWER
I20260807 18:33:12.353335 13766 consensus_queue.cc:260] T c36b772e33454c9aaf039ceebfc1177c P 97e3153b9b104e5a9fca75dcbb61571b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97e3153b9b104e5a9fca75dcbb61571b" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 45267 } } peers { permanent_uuid: "3596f9392f8746a09722d927914e3f20" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 41295 } } peers { permanent_uuid: "99ba8d46c261419aa0813e9e10dc6cd1" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 45171 } }
I20260807 18:33:12.354009 13444 heartbeater.cc:503] Master 127.11.14.126:36947 was elected leader, sending a full tablet report...
I20260807 18:33:12.354351 13766 ts_tablet_manager.cc:1428] T c36b772e33454c9aaf039ceebfc1177c P 97e3153b9b104e5a9fca75dcbb61571b: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260807 18:33:12.355746 13767 raft_consensus.cc:348] T c36b772e33454c9aaf039ceebfc1177c P 3596f9392f8746a09722d927914e3f20 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97e3153b9b104e5a9fca75dcbb61571b" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 45267 } } peers { permanent_uuid: "3596f9392f8746a09722d927914e3f20" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 41295 } } peers { permanent_uuid: "99ba8d46c261419aa0813e9e10dc6cd1" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 45171 } }
I20260807 18:33:12.355978 13767 raft_consensus.cc:374] T c36b772e33454c9aaf039ceebfc1177c P 3596f9392f8746a09722d927914e3f20 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:33:12.356042 13767 raft_consensus.cc:729] T c36b772e33454c9aaf039ceebfc1177c P 3596f9392f8746a09722d927914e3f20 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3596f9392f8746a09722d927914e3f20, State: Initialized, Role: FOLLOWER
I20260807 18:33:12.356180 13771 tablet_bootstrap.cc:654] T c36b772e33454c9aaf039ceebfc1177c P 99ba8d46c261419aa0813e9e10dc6cd1: Neither blocks nor log segments found. Creating new log.
I20260807 18:33:12.356540 13767 consensus_queue.cc:260] T c36b772e33454c9aaf039ceebfc1177c P 3596f9392f8746a09722d927914e3f20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97e3153b9b104e5a9fca75dcbb61571b" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 45267 } } peers { permanent_uuid: "3596f9392f8746a09722d927914e3f20" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 41295 } } peers { permanent_uuid: "99ba8d46c261419aa0813e9e10dc6cd1" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 45171 } }
I20260807 18:33:12.357126 13771 log.cc:824] T c36b772e33454c9aaf039ceebfc1177c P 99ba8d46c261419aa0813e9e10dc6cd1: Log is configured to *not* fsync() on all Append() calls
I20260807 18:33:12.357252 13741 heartbeater.cc:503] Master 127.11.14.126:36947 was elected leader, sending a full tablet report...
I20260807 18:33:12.358959 13767 ts_tablet_manager.cc:1428] T c36b772e33454c9aaf039ceebfc1177c P 3596f9392f8746a09722d927914e3f20: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260807 18:33:12.359362 13771 tablet_bootstrap.cc:492] T c36b772e33454c9aaf039ceebfc1177c P 99ba8d46c261419aa0813e9e10dc6cd1: No bootstrap required, opened a new log
I20260807 18:33:12.359622 13771 ts_tablet_manager.cc:1397] T c36b772e33454c9aaf039ceebfc1177c P 99ba8d46c261419aa0813e9e10dc6cd1: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260807 18:33:12.363395 13771 raft_consensus.cc:348] T c36b772e33454c9aaf039ceebfc1177c P 99ba8d46c261419aa0813e9e10dc6cd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97e3153b9b104e5a9fca75dcbb61571b" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 45267 } } peers { permanent_uuid: "3596f9392f8746a09722d927914e3f20" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 41295 } } peers { permanent_uuid: "99ba8d46c261419aa0813e9e10dc6cd1" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 45171 } }
I20260807 18:33:12.363682 13771 raft_consensus.cc:374] T c36b772e33454c9aaf039ceebfc1177c P 99ba8d46c261419aa0813e9e10dc6cd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:33:12.363724 13771 raft_consensus.cc:729] T c36b772e33454c9aaf039ceebfc1177c P 99ba8d46c261419aa0813e9e10dc6cd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99ba8d46c261419aa0813e9e10dc6cd1, State: Initialized, Role: FOLLOWER
I20260807 18:33:12.364120 13771 consensus_queue.cc:260] T c36b772e33454c9aaf039ceebfc1177c P 99ba8d46c261419aa0813e9e10dc6cd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97e3153b9b104e5a9fca75dcbb61571b" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 45267 } } peers { permanent_uuid: "3596f9392f8746a09722d927914e3f20" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 41295 } } peers { permanent_uuid: "99ba8d46c261419aa0813e9e10dc6cd1" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 45171 } }
I20260807 18:33:12.364805 13591 heartbeater.cc:503] Master 127.11.14.126:36947 was elected leader, sending a full tablet report...
I20260807 18:33:12.365073 13771 ts_tablet_manager.cc:1428] T c36b772e33454c9aaf039ceebfc1177c P 99ba8d46c261419aa0813e9e10dc6cd1: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
W20260807 18:33:12.505486 13445 tablet.cc:2369] T c36b772e33454c9aaf039ceebfc1177c P 97e3153b9b104e5a9fca75dcbb61571b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 18:33:12.512804 13592 tablet.cc:2369] T c36b772e33454c9aaf039ceebfc1177c P 99ba8d46c261419aa0813e9e10dc6cd1: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 18:33:12.513833 13742 tablet.cc:2369] T c36b772e33454c9aaf039ceebfc1177c P 3596f9392f8746a09722d927914e3f20: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:33:12.566257 13773 raft_consensus.cc:482] T c36b772e33454c9aaf039ceebfc1177c P 3596f9392f8746a09722d927914e3f20 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:33:12.566499 13773 raft_consensus.cc:504] T c36b772e33454c9aaf039ceebfc1177c P 3596f9392f8746a09722d927914e3f20 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97e3153b9b104e5a9fca75dcbb61571b" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 45267 } } peers { permanent_uuid: "3596f9392f8746a09722d927914e3f20" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 41295 } } peers { permanent_uuid: "99ba8d46c261419aa0813e9e10dc6cd1" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 45171 } }
I20260807 18:33:12.567979 13773 leader_election.cc:288] T c36b772e33454c9aaf039ceebfc1177c P 3596f9392f8746a09722d927914e3f20 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97e3153b9b104e5a9fca75dcbb61571b (127.11.14.65:45267), 99ba8d46c261419aa0813e9e10dc6cd1 (127.11.14.66:45171)
I20260807 18:33:12.572472 13765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:12.568372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.67:53021 (local address 127.11.14.66:45171)
0807 18:33:12.568510 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:12.568515 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:12.568534 (+ 19us) server_negotiation.cc:407] Connection header received
0807 18:33:12.568672 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:12.568675 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:12.568731 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:12.568829 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:33:12.569162 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:12.569999 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:12.571312 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:12.571953 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:12.572045 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:12.572088 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:12.572310 (+ 222us) server_negotiation.cc:299] Negotiation successful
0807 18:33:12.572368 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260807 18:33:12.572494 13626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:12.568278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.66:45171 (local address 127.11.14.67:53021)
0807 18:33:12.568424 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:12.568442 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:12.568540 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:12.568871 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:12.568876 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:12.568901 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:33:12.569045 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:12.569054 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:12.570105 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:12.570109 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:12.571186 (+ 1077us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:12.571195 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:12.572060 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:12.572064 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:12.572180 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:12.572206 (+ 26us) client_negotiation.cc:769] Sending connection context
0807 18:33:12.572286 (+ 80us) client_negotiation.cc:240] Negotiation successful
0807 18:33:12.572356 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":59}
I20260807 18:33:12.573210 13776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:12.568303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.65:45267 (local address 127.11.14.67:37065)
0807 18:33:12.568757 (+ 454us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:12.568773 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:12.568873 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:12.569157 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:12.569160 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:12.569178 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:33:12.569806 (+ 628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:12.569817 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:12.570806 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:12.570809 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:12.571794 (+ 985us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:12.571806 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:12.572770 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:12.572773 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:12.572890 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:12.572909 (+ 19us) client_negotiation.cc:769] Sending connection context
0807 18:33:12.573007 (+ 98us) client_negotiation.cc:240] Negotiation successful
0807 18:33:12.573078 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":386,"thread_start_us":120,"threads_started":1}
I20260807 18:33:12.573227 13761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:12.568536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.67:37065 (local address 127.11.14.65:45267)
0807 18:33:12.568677 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:12.568681 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:12.568857 (+ 176us) server_negotiation.cc:407] Connection header received
0807 18:33:12.568977 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:12.568980 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:12.569032 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:12.569120 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:33:12.569928 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:12.570688 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:12.571917 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:12.572570 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:12.572690 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:12.572751 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:12.573044 (+ 293us) server_negotiation.cc:299] Negotiation successful
0807 18:33:12.573118 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260807 18:33:12.573233 13541 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c36b772e33454c9aaf039ceebfc1177c" candidate_uuid: "3596f9392f8746a09722d927914e3f20" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99ba8d46c261419aa0813e9e10dc6cd1" is_pre_election: true
I20260807 18:33:12.573629 13541 raft_consensus.cc:2454] T c36b772e33454c9aaf039ceebfc1177c P 99ba8d46c261419aa0813e9e10dc6cd1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3596f9392f8746a09722d927914e3f20 in term 0.
I20260807 18:33:12.573927 13392 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c36b772e33454c9aaf039ceebfc1177c" candidate_uuid: "3596f9392f8746a09722d927914e3f20" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97e3153b9b104e5a9fca75dcbb61571b" is_pre_election: true
I20260807 18:33:12.574193 13618 leader_election.cc:302] T c36b772e33454c9aaf039ceebfc1177c P 3596f9392f8746a09722d927914e3f20 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3596f9392f8746a09722d927914e3f20, 99ba8d46c261419aa0813e9e10dc6cd1; no voters:
I20260807 18:33:12.574287 13392 raft_consensus.cc:2454] T c36b772e33454c9aaf039ceebfc1177c P 97e3153b9b104e5a9fca75dcbb61571b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3596f9392f8746a09722d927914e3f20 in term 0.
I20260807 18:33:12.574489 13773 raft_consensus.cc:2781] T c36b772e33454c9aaf039ceebfc1177c P 3596f9392f8746a09722d927914e3f20 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:33:12.574594 13773 raft_consensus.cc:482] T c36b772e33454c9aaf039ceebfc1177c P 3596f9392f8746a09722d927914e3f20 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:33:12.574658 13773 raft_consensus.cc:3037] T c36b772e33454c9aaf039ceebfc1177c P 3596f9392f8746a09722d927914e3f20 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:33:12.575955 13773 raft_consensus.cc:504] T c36b772e33454c9aaf039ceebfc1177c P 3596f9392f8746a09722d927914e3f20 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97e3153b9b104e5a9fca75dcbb61571b" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 45267 } } peers { permanent_uuid: "3596f9392f8746a09722d927914e3f20" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 41295 } } peers { permanent_uuid: "99ba8d46c261419aa0813e9e10dc6cd1" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 45171 } }
I20260807 18:33:12.576310 13773 leader_election.cc:288] T c36b772e33454c9aaf039ceebfc1177c P 3596f9392f8746a09722d927914e3f20 [CANDIDATE]: Term 1 election: Requested vote from peers 97e3153b9b104e5a9fca75dcbb61571b (127.11.14.65:45267), 99ba8d46c261419aa0813e9e10dc6cd1 (127.11.14.66:45171)
I20260807 18:33:12.576674 13541 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c36b772e33454c9aaf039ceebfc1177c" candidate_uuid: "3596f9392f8746a09722d927914e3f20" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99ba8d46c261419aa0813e9e10dc6cd1"
I20260807 18:33:12.576732 13392 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c36b772e33454c9aaf039ceebfc1177c" candidate_uuid: "3596f9392f8746a09722d927914e3f20" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97e3153b9b104e5a9fca75dcbb61571b"
I20260807 18:33:12.576826 13541 raft_consensus.cc:3037] T c36b772e33454c9aaf039ceebfc1177c P 99ba8d46c261419aa0813e9e10dc6cd1 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:33:12.576879 13392 raft_consensus.cc:3037] T c36b772e33454c9aaf039ceebfc1177c P 97e3153b9b104e5a9fca75dcbb61571b [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:33:12.577955 13541 raft_consensus.cc:2454] T c36b772e33454c9aaf039ceebfc1177c P 99ba8d46c261419aa0813e9e10dc6cd1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3596f9392f8746a09722d927914e3f20 in term 1.
I20260807 18:33:12.577961 13392 raft_consensus.cc:2454] T c36b772e33454c9aaf039ceebfc1177c P 97e3153b9b104e5a9fca75dcbb61571b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3596f9392f8746a09722d927914e3f20 in term 1.
I20260807 18:33:12.578274 13618 leader_election.cc:302] T c36b772e33454c9aaf039ceebfc1177c P 3596f9392f8746a09722d927914e3f20 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3596f9392f8746a09722d927914e3f20, 99ba8d46c261419aa0813e9e10dc6cd1; no voters:
I20260807 18:33:12.578473 13773 raft_consensus.cc:2781] T c36b772e33454c9aaf039ceebfc1177c P 3596f9392f8746a09722d927914e3f20 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:33:12.578671 13773 raft_consensus.cc:686] T c36b772e33454c9aaf039ceebfc1177c P 3596f9392f8746a09722d927914e3f20 [term 1 LEADER]: Becoming Leader. State: Replica: 3596f9392f8746a09722d927914e3f20, State: Running, Role: LEADER
I20260807 18:33:12.579005 13773 consensus_queue.cc:237] T c36b772e33454c9aaf039ceebfc1177c P 3596f9392f8746a09722d927914e3f20 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97e3153b9b104e5a9fca75dcbb61571b" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 45267 } } peers { permanent_uuid: "3596f9392f8746a09722d927914e3f20" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 41295 } } peers { permanent_uuid: "99ba8d46c261419aa0813e9e10dc6cd1" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 45171 } }
I20260807 18:33:12.582423 13221 catalog_manager.cc:5495] T c36b772e33454c9aaf039ceebfc1177c P 3596f9392f8746a09722d927914e3f20 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3596f9392f8746a09722d927914e3f20 (127.11.14.67). New cstate: current_term: 1 leader_uuid: "3596f9392f8746a09722d927914e3f20" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97e3153b9b104e5a9fca75dcbb61571b" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 45267 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3596f9392f8746a09722d927914e3f20" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 41295 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "99ba8d46c261419aa0813e9e10dc6cd1" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 45171 } health_report { overall_health: UNKNOWN } } }
Aug 07 18:33:12 dist-test-slave-6k22 krb5kdc[13174](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127591, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.126@KRBTEST.COM for kudu/127.11.14.126@KRBTEST.COM
I20260807 18:33:12.649951 13787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:12.641404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:36947 (local address 127.0.0.1:54836)
0807 18:33:12.641758 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:12.641771 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:12.641912 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:12.642176 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:12.642179 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:12.642627 (+ 448us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:33:12.642736 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:12.642743 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:12.643496 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:12.643503 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:12.644133 (+ 630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:12.644142 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:12.644579 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:12.644582 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:12.644655 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:12.645336 (+ 681us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:33:12.645351 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:33:12.646792 (+ 1441us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:33:12.648296 (+ 1504us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:33:12.648308 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:33:12.648316 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:33:12.648547 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:33:12.648818 (+ 271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:33:12.648822 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:33:12.648824 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:33:12.648908 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:33:12.649296 (+ 388us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:33:12.649306 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:33:12.649434 (+ 128us) client_negotiation.cc:769] Sending connection context
0807 18:33:12.649552 (+ 118us) client_negotiation.cc:240] Negotiation successful
0807 18:33:12.649737 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":227,"thread_start_us":104,"threads_started":1}
I20260807 18:33:12.649951 13277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:12.641507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54836 (local address 127.11.14.126:36947)
0807 18:33:12.641646 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:12.641652 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:12.641855 (+ 203us) server_negotiation.cc:407] Connection header received
0807 18:33:12.642019 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:12.642022 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:12.642064 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:12.642156 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:33:12.642866 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:12.643386 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:12.644262 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:12.644469 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:12.644579 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:12.646936 (+ 2357us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:33:12.646939 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:33:12.646941 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:33:12.646961 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:33:12.648182 (+ 1221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:12.648649 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:12.648654 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:12.648656 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:12.648728 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:12.649007 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:12.649012 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:12.649013 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:12.649179 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:33:12.649295 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:12.649600 (+ 305us) server_negotiation.cc:299] Negotiation successful
0807 18:33:12.649792 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":93,"server-negotiator.queue_time_us":51}
I20260807 18:33:12.653995 13221 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.11.14.126@KRBTEST.COM'} at 127.0.0.1:54836:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260807 18:33:12.654675 13221 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 18:33:12.659943 13371 tablet_service.cc:1467] Processing CreateTablet for tablet c736643195914c06b4d7db61ff247564 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=8d688fe083794f7995a21c22ab5a105f]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 18:33:12.660125 13673 tablet_service.cc:1467] Processing CreateTablet for tablet c736643195914c06b4d7db61ff247564 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=8d688fe083794f7995a21c22ab5a105f]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 18:33:12.660048 13515 tablet_service.cc:1467] Processing CreateTablet for tablet c736643195914c06b4d7db61ff247564 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=8d688fe083794f7995a21c22ab5a105f]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 18:33:12.660701 13371 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c736643195914c06b4d7db61ff247564. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:33:12.660758 13515 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c736643195914c06b4d7db61ff247564. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:33:12.660758 13673 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c736643195914c06b4d7db61ff247564. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:33:12.663141 13766 tablet_bootstrap.cc:492] T c736643195914c06b4d7db61ff247564 P 97e3153b9b104e5a9fca75dcbb61571b: Bootstrap starting.
I20260807 18:33:12.663762 13767 tablet_bootstrap.cc:492] T c736643195914c06b4d7db61ff247564 P 3596f9392f8746a09722d927914e3f20: Bootstrap starting.
I20260807 18:33:12.664216 13766 tablet_bootstrap.cc:654] T c736643195914c06b4d7db61ff247564 P 97e3153b9b104e5a9fca75dcbb61571b: Neither blocks nor log segments found. Creating new log.
I20260807 18:33:12.664690 13767 tablet_bootstrap.cc:654] T c736643195914c06b4d7db61ff247564 P 3596f9392f8746a09722d927914e3f20: Neither blocks nor log segments found. Creating new log.
I20260807 18:33:12.665391 13771 tablet_bootstrap.cc:492] T c736643195914c06b4d7db61ff247564 P 99ba8d46c261419aa0813e9e10dc6cd1: Bootstrap starting.
I20260807 18:33:12.665644 13766 tablet_bootstrap.cc:492] T c736643195914c06b4d7db61ff247564 P 97e3153b9b104e5a9fca75dcbb61571b: No bootstrap required, opened a new log
I20260807 18:33:12.665761 13766 ts_tablet_manager.cc:1397] T c736643195914c06b4d7db61ff247564 P 97e3153b9b104e5a9fca75dcbb61571b: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260807 18:33:12.666074 13767 tablet_bootstrap.cc:492] T c736643195914c06b4d7db61ff247564 P 3596f9392f8746a09722d927914e3f20: No bootstrap required, opened a new log
I20260807 18:33:12.666167 13767 ts_tablet_manager.cc:1397] T c736643195914c06b4d7db61ff247564 P 3596f9392f8746a09722d927914e3f20: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260807 18:33:12.666211 13766 raft_consensus.cc:348] T c736643195914c06b4d7db61ff247564 P 97e3153b9b104e5a9fca75dcbb61571b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97e3153b9b104e5a9fca75dcbb61571b" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 45267 } } peers { permanent_uuid: "99ba8d46c261419aa0813e9e10dc6cd1" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 45171 } } peers { permanent_uuid: "3596f9392f8746a09722d927914e3f20" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 41295 } }
I20260807 18:33:12.666354 13766 raft_consensus.cc:374] T c736643195914c06b4d7db61ff247564 P 97e3153b9b104e5a9fca75dcbb61571b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:33:12.666393 13766 raft_consensus.cc:729] T c736643195914c06b4d7db61ff247564 P 97e3153b9b104e5a9fca75dcbb61571b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97e3153b9b104e5a9fca75dcbb61571b, State: Initialized, Role: FOLLOWER
I20260807 18:33:12.666433 13771 tablet_bootstrap.cc:654] T c736643195914c06b4d7db61ff247564 P 99ba8d46c261419aa0813e9e10dc6cd1: Neither blocks nor log segments found. Creating new log.
I20260807 18:33:12.666560 13766 consensus_queue.cc:260] T c736643195914c06b4d7db61ff247564 P 97e3153b9b104e5a9fca75dcbb61571b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97e3153b9b104e5a9fca75dcbb61571b" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 45267 } } peers { permanent_uuid: "99ba8d46c261419aa0813e9e10dc6cd1" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 45171 } } peers { permanent_uuid: "3596f9392f8746a09722d927914e3f20" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 41295 } }
I20260807 18:33:12.666679 13767 raft_consensus.cc:348] T c736643195914c06b4d7db61ff247564 P 3596f9392f8746a09722d927914e3f20 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97e3153b9b104e5a9fca75dcbb61571b" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 45267 } } peers { permanent_uuid: "99ba8d46c261419aa0813e9e10dc6cd1" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 45171 } } peers { permanent_uuid: "3596f9392f8746a09722d927914e3f20" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 41295 } }
I20260807 18:33:12.666810 13767 raft_consensus.cc:374] T c736643195914c06b4d7db61ff247564 P 3596f9392f8746a09722d927914e3f20 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:33:12.666857 13767 raft_consensus.cc:729] T c736643195914c06b4d7db61ff247564 P 3596f9392f8746a09722d927914e3f20 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3596f9392f8746a09722d927914e3f20, State: Initialized, Role: FOLLOWER
I20260807 18:33:12.666833 13772 tablet_replica.cc:439] T c736643195914c06b4d7db61ff247564 P 97e3153b9b104e5a9fca75dcbb61571b: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97e3153b9b104e5a9fca75dcbb61571b" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 45267 } } peers { permanent_uuid: "99ba8d46c261419aa0813e9e10dc6cd1" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 45171 } } peers { permanent_uuid: "3596f9392f8746a09722d927914e3f20" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 41295 } } }
I20260807 18:33:12.666976 13772 tablet_replica.cc:442] T c736643195914c06b4d7db61ff247564 P 97e3153b9b104e5a9fca75dcbb61571b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 18:33:12.666990 13767 consensus_queue.cc:260] T c736643195914c06b4d7db61ff247564 P 3596f9392f8746a09722d927914e3f20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97e3153b9b104e5a9fca75dcbb61571b" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 45267 } } peers { permanent_uuid: "99ba8d46c261419aa0813e9e10dc6cd1" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 45171 } } peers { permanent_uuid: "3596f9392f8746a09722d927914e3f20" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 41295 } }
I20260807 18:33:12.667203 13766 ts_tablet_manager.cc:1428] T c736643195914c06b4d7db61ff247564 P 97e3153b9b104e5a9fca75dcbb61571b: Time spent starting tablet: real 0.001s user 0.000s sys 0.002s
I20260807 18:33:12.667265 13773 tablet_replica.cc:439] T c736643195914c06b4d7db61ff247564 P 3596f9392f8746a09722d927914e3f20: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97e3153b9b104e5a9fca75dcbb61571b" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 45267 } } peers { permanent_uuid: "99ba8d46c261419aa0813e9e10dc6cd1" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 45171 } } peers { permanent_uuid: "3596f9392f8746a09722d927914e3f20" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 41295 } } }
I20260807 18:33:12.667397 13773 tablet_replica.cc:442] T c736643195914c06b4d7db61ff247564 P 3596f9392f8746a09722d927914e3f20: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 18:33:12.668059 13767 ts_tablet_manager.cc:1428] T c736643195914c06b4d7db61ff247564 P 3596f9392f8746a09722d927914e3f20: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260807 18:33:12.669386 13771 tablet_bootstrap.cc:492] T c736643195914c06b4d7db61ff247564 P 99ba8d46c261419aa0813e9e10dc6cd1: No bootstrap required, opened a new log
I20260807 18:33:12.669489 13771 ts_tablet_manager.cc:1397] T c736643195914c06b4d7db61ff247564 P 99ba8d46c261419aa0813e9e10dc6cd1: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260807 18:33:12.670004 13771 raft_consensus.cc:348] T c736643195914c06b4d7db61ff247564 P 99ba8d46c261419aa0813e9e10dc6cd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97e3153b9b104e5a9fca75dcbb61571b" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 45267 } } peers { permanent_uuid: "99ba8d46c261419aa0813e9e10dc6cd1" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 45171 } } peers { permanent_uuid: "3596f9392f8746a09722d927914e3f20" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 41295 } }
I20260807 18:33:12.670130 13771 raft_consensus.cc:374] T c736643195914c06b4d7db61ff247564 P 99ba8d46c261419aa0813e9e10dc6cd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:33:12.670176 13771 raft_consensus.cc:729] T c736643195914c06b4d7db61ff247564 P 99ba8d46c261419aa0813e9e10dc6cd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99ba8d46c261419aa0813e9e10dc6cd1, State: Initialized, Role: FOLLOWER
I20260807 18:33:12.670378 13771 consensus_queue.cc:260] T c736643195914c06b4d7db61ff247564 P 99ba8d46c261419aa0813e9e10dc6cd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97e3153b9b104e5a9fca75dcbb61571b" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 45267 } } peers { permanent_uuid: "99ba8d46c261419aa0813e9e10dc6cd1" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 45171 } } peers { permanent_uuid: "3596f9392f8746a09722d927914e3f20" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 41295 } }
I20260807 18:33:12.670670 13771 ts_tablet_manager.cc:1428] T c736643195914c06b4d7db61ff247564 P 99ba8d46c261419aa0813e9e10dc6cd1: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260807 18:33:12.670648 13775 tablet_replica.cc:439] T c736643195914c06b4d7db61ff247564 P 99ba8d46c261419aa0813e9e10dc6cd1: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97e3153b9b104e5a9fca75dcbb61571b" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 45267 } } peers { permanent_uuid: "99ba8d46c261419aa0813e9e10dc6cd1" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 45171 } } peers { permanent_uuid: "3596f9392f8746a09722d927914e3f20" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 41295 } } }
I20260807 18:33:12.671120 13775 tablet_replica.cc:442] T c736643195914c06b4d7db61ff247564 P 99ba8d46c261419aa0813e9e10dc6cd1: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 18:33:12.995317 13773 consensus_queue.cc:1035] T c36b772e33454c9aaf039ceebfc1177c P 3596f9392f8746a09722d927914e3f20 [LEADER]: Connected to new peer: Peer: permanent_uuid: "97e3153b9b104e5a9fca75dcbb61571b" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 45267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:33:13.001467 13773 consensus_queue.cc:1035] T c36b772e33454c9aaf039ceebfc1177c P 3596f9392f8746a09722d927914e3f20 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99ba8d46c261419aa0813e9e10dc6cd1" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 45171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:33:13.003057 13773 raft_consensus.cc:482] T c736643195914c06b4d7db61ff247564 P 3596f9392f8746a09722d927914e3f20 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:33:13.003194 13773 raft_consensus.cc:504] T c736643195914c06b4d7db61ff247564 P 3596f9392f8746a09722d927914e3f20 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97e3153b9b104e5a9fca75dcbb61571b" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 45267 } } peers { permanent_uuid: "99ba8d46c261419aa0813e9e10dc6cd1" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 45171 } } peers { permanent_uuid: "3596f9392f8746a09722d927914e3f20" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 41295 } }
I20260807 18:33:13.003646 13773 leader_election.cc:288] T c736643195914c06b4d7db61ff247564 P 3596f9392f8746a09722d927914e3f20 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97e3153b9b104e5a9fca75dcbb61571b (127.11.14.65:45267), 99ba8d46c261419aa0813e9e10dc6cd1 (127.11.14.66:45171)
I20260807 18:33:13.004092 13392 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c736643195914c06b4d7db61ff247564" candidate_uuid: "3596f9392f8746a09722d927914e3f20" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97e3153b9b104e5a9fca75dcbb61571b" is_pre_election: true
I20260807 18:33:13.004086 13540 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c736643195914c06b4d7db61ff247564" candidate_uuid: "3596f9392f8746a09722d927914e3f20" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99ba8d46c261419aa0813e9e10dc6cd1" is_pre_election: true
I20260807 18:33:13.004225 13392 raft_consensus.cc:2454] T c736643195914c06b4d7db61ff247564 P 97e3153b9b104e5a9fca75dcbb61571b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3596f9392f8746a09722d927914e3f20 in term 0.
I20260807 18:33:13.004242 13540 raft_consensus.cc:2454] T c736643195914c06b4d7db61ff247564 P 99ba8d46c261419aa0813e9e10dc6cd1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3596f9392f8746a09722d927914e3f20 in term 0.
I20260807 18:33:13.004499 13618 leader_election.cc:302] T c736643195914c06b4d7db61ff247564 P 3596f9392f8746a09722d927914e3f20 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3596f9392f8746a09722d927914e3f20, 99ba8d46c261419aa0813e9e10dc6cd1; no voters:
I20260807 18:33:13.004685 13773 raft_consensus.cc:2781] T c736643195914c06b4d7db61ff247564 P 3596f9392f8746a09722d927914e3f20 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:33:13.004750 13773 raft_consensus.cc:482] T c736643195914c06b4d7db61ff247564 P 3596f9392f8746a09722d927914e3f20 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:33:13.004796 13773 raft_consensus.cc:3037] T c736643195914c06b4d7db61ff247564 P 3596f9392f8746a09722d927914e3f20 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:33:13.005760 13773 raft_consensus.cc:504] T c736643195914c06b4d7db61ff247564 P 3596f9392f8746a09722d927914e3f20 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97e3153b9b104e5a9fca75dcbb61571b" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 45267 } } peers { permanent_uuid: "99ba8d46c261419aa0813e9e10dc6cd1" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 45171 } } peers { permanent_uuid: "3596f9392f8746a09722d927914e3f20" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 41295 } }
I20260807 18:33:13.006071 13773 leader_election.cc:288] T c736643195914c06b4d7db61ff247564 P 3596f9392f8746a09722d927914e3f20 [CANDIDATE]: Term 1 election: Requested vote from peers 97e3153b9b104e5a9fca75dcbb61571b (127.11.14.65:45267), 99ba8d46c261419aa0813e9e10dc6cd1 (127.11.14.66:45171)
I20260807 18:33:13.006318 13392 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c736643195914c06b4d7db61ff247564" candidate_uuid: "3596f9392f8746a09722d927914e3f20" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97e3153b9b104e5a9fca75dcbb61571b"
I20260807 18:33:13.006455 13392 raft_consensus.cc:3037] T c736643195914c06b4d7db61ff247564 P 97e3153b9b104e5a9fca75dcbb61571b [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:33:13.007025 13541 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c736643195914c06b4d7db61ff247564" candidate_uuid: "3596f9392f8746a09722d927914e3f20" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99ba8d46c261419aa0813e9e10dc6cd1"
I20260807 18:33:13.007153 13541 raft_consensus.cc:3037] T c736643195914c06b4d7db61ff247564 P 99ba8d46c261419aa0813e9e10dc6cd1 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:33:13.007539 13392 raft_consensus.cc:2454] T c736643195914c06b4d7db61ff247564 P 97e3153b9b104e5a9fca75dcbb61571b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3596f9392f8746a09722d927914e3f20 in term 1.
I20260807 18:33:13.008011 13617 leader_election.cc:302] T c736643195914c06b4d7db61ff247564 P 3596f9392f8746a09722d927914e3f20 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3596f9392f8746a09722d927914e3f20, 97e3153b9b104e5a9fca75dcbb61571b; no voters:
I20260807 18:33:13.008198 13773 raft_consensus.cc:2781] T c736643195914c06b4d7db61ff247564 P 3596f9392f8746a09722d927914e3f20 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:33:13.008272 13541 raft_consensus.cc:2454] T c736643195914c06b4d7db61ff247564 P 99ba8d46c261419aa0813e9e10dc6cd1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3596f9392f8746a09722d927914e3f20 in term 1.
I20260807 18:33:13.008302 13773 raft_consensus.cc:686] T c736643195914c06b4d7db61ff247564 P 3596f9392f8746a09722d927914e3f20 [term 1 LEADER]: Becoming Leader. State: Replica: 3596f9392f8746a09722d927914e3f20, State: Running, Role: LEADER
I20260807 18:33:13.008440 13773 consensus_queue.cc:237] T c736643195914c06b4d7db61ff247564 P 3596f9392f8746a09722d927914e3f20 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97e3153b9b104e5a9fca75dcbb61571b" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 45267 } } peers { permanent_uuid: "99ba8d46c261419aa0813e9e10dc6cd1" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 45171 } } peers { permanent_uuid: "3596f9392f8746a09722d927914e3f20" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 41295 } }
I20260807 18:33:13.009167 13794 tablet_replica.cc:439] T c736643195914c06b4d7db61ff247564 P 3596f9392f8746a09722d927914e3f20: TxnStatusTablet state changed. Reason: New leader 3596f9392f8746a09722d927914e3f20. Latest consensus state: current_term: 1 leader_uuid: "3596f9392f8746a09722d927914e3f20" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97e3153b9b104e5a9fca75dcbb61571b" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 45267 } } peers { permanent_uuid: "99ba8d46c261419aa0813e9e10dc6cd1" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 45171 } } peers { permanent_uuid: "3596f9392f8746a09722d927914e3f20" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 41295 } } }
I20260807 18:33:13.009284 13794 tablet_replica.cc:442] T c736643195914c06b4d7db61ff247564 P 3596f9392f8746a09722d927914e3f20: This TxnStatusTablet replica's current role is: LEADER
I20260807 18:33:13.009531 13800 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260807 18:33:13.009756 13800 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260807 18:33:13.010079 13221 catalog_manager.cc:5495] T c736643195914c06b4d7db61ff247564 P 3596f9392f8746a09722d927914e3f20 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3596f9392f8746a09722d927914e3f20 (127.11.14.67). New cstate: current_term: 1 leader_uuid: "3596f9392f8746a09722d927914e3f20" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97e3153b9b104e5a9fca75dcbb61571b" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 45267 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "99ba8d46c261419aa0813e9e10dc6cd1" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 45171 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3596f9392f8746a09722d927914e3f20" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 41295 } health_report { overall_health: HEALTHY } } }
I20260807 18:33:13.036967 13803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:13.032891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50114 (local address 127.11.14.67:41295)
0807 18:33:13.033258 (+ 367us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:13.033266 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:13.033278 (+ 12us) server_negotiation.cc:407] Connection header received
0807 18:33:13.033365 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:13.033367 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:13.033421 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:13.033544 (+ 123us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:33:13.033838 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:13.034402 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:13.035276 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:13.035518 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:13.035683 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:13.036040 (+ 357us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:33:13.036501 (+ 461us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:33:13.036630 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:13.036706 (+ 76us) server_negotiation.cc:299] Negotiation successful
0807 18:33:13.036760 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":275,"thread_start_us":133,"threads_started":1}
I20260807 18:33:13.036947 13787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:13.032758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.67:41295 (local address 127.0.0.1:50114)
0807 18:33:13.032953 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:13.032978 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:13.033100 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:13.033561 (+ 461us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:13.033564 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:13.033572 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 18:33:13.033708 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:13.033713 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:13.034523 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:13.034528 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:13.035145 (+ 617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:13.035154 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:13.035698 (+ 544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:13.035701 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:13.035797 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:13.035852 (+ 55us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 18:33:13.036604 (+ 752us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 18:33:13.036622 (+ 18us) client_negotiation.cc:769] Sending connection context
0807 18:33:13.036694 (+ 72us) client_negotiation.cc:240] Negotiation successful
0807 18:33:13.036782 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":59}
I20260807 18:33:13.041885 13541 raft_consensus.cc:1261] T c736643195914c06b4d7db61ff247564 P 99ba8d46c261419aa0813e9e10dc6cd1 [term 1 FOLLOWER]: Refusing update from remote peer 3596f9392f8746a09722d927914e3f20: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:33:13.041916 13392 raft_consensus.cc:1261] T c736643195914c06b4d7db61ff247564 P 97e3153b9b104e5a9fca75dcbb61571b [term 1 FOLLOWER]: Refusing update from remote peer 3596f9392f8746a09722d927914e3f20: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:33:13.042388 13773 consensus_queue.cc:1035] T c736643195914c06b4d7db61ff247564 P 3596f9392f8746a09722d927914e3f20 [LEADER]: Connected to new peer: Peer: permanent_uuid: "97e3153b9b104e5a9fca75dcbb61571b" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 45267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:33:13.042570 13794 consensus_queue.cc:1035] T c736643195914c06b4d7db61ff247564 P 3596f9392f8746a09722d927914e3f20 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99ba8d46c261419aa0813e9e10dc6cd1" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 45171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:33:13.045511 13772 tablet_replica.cc:439] T c736643195914c06b4d7db61ff247564 P 97e3153b9b104e5a9fca75dcbb61571b: TxnStatusTablet state changed. Reason: New leader 3596f9392f8746a09722d927914e3f20. Latest consensus state: current_term: 1 leader_uuid: "3596f9392f8746a09722d927914e3f20" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97e3153b9b104e5a9fca75dcbb61571b" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 45267 } } peers { permanent_uuid: "99ba8d46c261419aa0813e9e10dc6cd1" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 45171 } } peers { permanent_uuid: "3596f9392f8746a09722d927914e3f20" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 41295 } } }
I20260807 18:33:13.045641 13772 tablet_replica.cc:442] T c736643195914c06b4d7db61ff247564 P 97e3153b9b104e5a9fca75dcbb61571b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 18:33:13.045826 13775 tablet_replica.cc:439] T c736643195914c06b4d7db61ff247564 P 99ba8d46c261419aa0813e9e10dc6cd1: TxnStatusTablet state changed. Reason: New leader 3596f9392f8746a09722d927914e3f20. Latest consensus state: current_term: 1 leader_uuid: "3596f9392f8746a09722d927914e3f20" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97e3153b9b104e5a9fca75dcbb61571b" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 45267 } } peers { permanent_uuid: "99ba8d46c261419aa0813e9e10dc6cd1" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 45171 } } peers { permanent_uuid: "3596f9392f8746a09722d927914e3f20" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 41295 } } }
I20260807 18:33:13.045969 13775 tablet_replica.cc:442] T c736643195914c06b4d7db61ff247564 P 99ba8d46c261419aa0813e9e10dc6cd1: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 18:33:13.049737 13778 tablet_replica.cc:439] T c736643195914c06b4d7db61ff247564 P 3596f9392f8746a09722d927914e3f20: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3596f9392f8746a09722d927914e3f20" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97e3153b9b104e5a9fca75dcbb61571b" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 45267 } } peers { permanent_uuid: "99ba8d46c261419aa0813e9e10dc6cd1" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 45171 } } peers { permanent_uuid: "3596f9392f8746a09722d927914e3f20" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 41295 } } }
I20260807 18:33:13.049886 13778 tablet_replica.cc:442] T c736643195914c06b4d7db61ff247564 P 3596f9392f8746a09722d927914e3f20: This TxnStatusTablet replica's current role is: LEADER
I20260807 18:33:13.049834 13794 tablet_replica.cc:439] T c736643195914c06b4d7db61ff247564 P 3596f9392f8746a09722d927914e3f20: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3596f9392f8746a09722d927914e3f20" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97e3153b9b104e5a9fca75dcbb61571b" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 45267 } } peers { permanent_uuid: "99ba8d46c261419aa0813e9e10dc6cd1" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 45171 } } peers { permanent_uuid: "3596f9392f8746a09722d927914e3f20" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 41295 } } }
I20260807 18:33:13.049940 13794 tablet_replica.cc:442] T c736643195914c06b4d7db61ff247564 P 3596f9392f8746a09722d927914e3f20: This TxnStatusTablet replica's current role is: LEADER
I20260807 18:33:13.050034 13793 mvcc.cc:205] Tried to move back new op lower bound from 7315978621093912576 to 7315978620964192256. Current Snapshot: MvccSnapshot[applied={T|T < 7315978621093912576}]
I20260807 18:33:13.050443 13797 mvcc.cc:205] Tried to move back new op lower bound from 7315978621093912576 to 7315978620964192256. Current Snapshot: MvccSnapshot[applied={T|T < 7315978621093912576}]
I20260807 18:33:13.050614 13775 tablet_replica.cc:439] T c736643195914c06b4d7db61ff247564 P 99ba8d46c261419aa0813e9e10dc6cd1: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3596f9392f8746a09722d927914e3f20" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97e3153b9b104e5a9fca75dcbb61571b" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 45267 } } peers { permanent_uuid: "99ba8d46c261419aa0813e9e10dc6cd1" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 45171 } } peers { permanent_uuid: "3596f9392f8746a09722d927914e3f20" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 41295 } } }
I20260807 18:33:13.050704 13775 tablet_replica.cc:442] T c736643195914c06b4d7db61ff247564 P 99ba8d46c261419aa0813e9e10dc6cd1: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 18:33:13.054322 13772 tablet_replica.cc:439] T c736643195914c06b4d7db61ff247564 P 97e3153b9b104e5a9fca75dcbb61571b: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3596f9392f8746a09722d927914e3f20" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97e3153b9b104e5a9fca75dcbb61571b" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 45267 } } peers { permanent_uuid: "99ba8d46c261419aa0813e9e10dc6cd1" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 45171 } } peers { permanent_uuid: "3596f9392f8746a09722d927914e3f20" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 41295 } } }
I20260807 18:33:13.054446 13772 tablet_replica.cc:442] T c736643195914c06b4d7db61ff247564 P 97e3153b9b104e5a9fca75dcbb61571b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 18:33:13.061198 13803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:13.057063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50116 (local address 127.11.14.67:41295)
0807 18:33:13.057215 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:13.057219 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:13.057389 (+ 170us) server_negotiation.cc:407] Connection header received
0807 18:33:13.057483 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:13.057486 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:13.057543 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:13.057631 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:33:13.057825 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:13.058729 (+ 904us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:13.060092 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:13.060349 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:13.060437 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:13.060724 (+ 287us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:33:13.060794 (+ 70us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:33:13.060857 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:13.060981 (+ 124us) server_negotiation.cc:299] Negotiation successful
0807 18:33:13.061027 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260807 18:33:13.073241 13814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:13.069277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.67:41295 (local address 127.11.14.67:35163)
0807 18:33:13.069615 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:13.069630 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:13.069739 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:13.069980 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:13.069983 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:13.069991 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 18:33:13.070150 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:13.070156 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:13.070826 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:13.070830 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:13.071861 (+ 1031us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:13.071871 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:13.072309 (+ 438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:13.072313 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:13.072416 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:13.072474 (+ 58us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 18:33:13.072831 (+ 357us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 18:33:13.072846 (+ 15us) client_negotiation.cc:769] Sending connection context
0807 18:33:13.072921 (+ 75us) client_negotiation.cc:240] Negotiation successful
0807 18:33:13.073010 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":233,"mutex_wait_us":37,"thread_start_us":126,"threads_started":1}
I20260807 18:33:13.073275 13803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:13.069339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.67:35163 (local address 127.11.14.67:41295)
0807 18:33:13.069472 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:13.069477 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:13.069704 (+ 227us) server_negotiation.cc:407] Connection header received
0807 18:33:13.069844 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:13.069847 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:13.069887 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:13.069949 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:33:13.070250 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:13.070725 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:13.071972 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:13.072174 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:13.072252 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:13.072636 (+ 384us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:33:13.072732 (+ 96us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:33:13.072798 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:13.072934 (+ 136us) server_negotiation.cc:299] Negotiation successful
0807 18:33:13.073004 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":5,"server-negotiator.queue_time_us":54}
I20260807 18:33:13.123884 13220 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54826:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:33:13.124094 13220 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54826:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:33:13.126739 13220 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e09adbeddf294788bec26d0474bcf37c: Sending DeleteTablet for 3 replicas of tablet c36b772e33454c9aaf039ceebfc1177c
I20260807 18:33:13.127647 13515 tablet_service.cc:1514] Processing DeleteTablet for tablet c36b772e33454c9aaf039ceebfc1177c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:33:13 UTC) from {username='kudu', principal='kudu/127.11.14.126@KRBTEST.COM'} at 127.0.0.1:55458
I20260807 18:33:13.127647 13672 tablet_service.cc:1514] Processing DeleteTablet for tablet c36b772e33454c9aaf039ceebfc1177c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:33:13 UTC) from {username='kudu', principal='kudu/127.11.14.126@KRBTEST.COM'} at 127.0.0.1:50110
I20260807 18:33:13.127825 13371 tablet_service.cc:1514] Processing DeleteTablet for tablet c36b772e33454c9aaf039ceebfc1177c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:33:13 UTC) from {username='kudu', principal='kudu/127.11.14.126@KRBTEST.COM'} at 127.0.0.1:36712
I20260807 18:33:13.128301 13818 tablet_replica.cc:330] T c36b772e33454c9aaf039ceebfc1177c P 97e3153b9b104e5a9fca75dcbb61571b: stopping tablet replica
I20260807 18:33:13.128633 13817 tablet_replica.cc:330] T c36b772e33454c9aaf039ceebfc1177c P 3596f9392f8746a09722d927914e3f20: stopping tablet replica
I20260807 18:33:13.128669 13818 raft_consensus.cc:2229] T c36b772e33454c9aaf039ceebfc1177c P 97e3153b9b104e5a9fca75dcbb61571b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:33:13.128765 13220 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54826:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 18:33:13.128893 13817 raft_consensus.cc:2229] T c36b772e33454c9aaf039ceebfc1177c P 3596f9392f8746a09722d927914e3f20 [term 1 LEADER]: Raft consensus shutting down.
I20260807 18:33:13.128911 13818 raft_consensus.cc:2258] T c36b772e33454c9aaf039ceebfc1177c P 97e3153b9b104e5a9fca75dcbb61571b [term 1 FOLLOWER]: Raft consensus is shut down!
W20260807 18:33:13.129160 13220 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 18:33:13.129245 13817 raft_consensus.cc:2258] T c36b772e33454c9aaf039ceebfc1177c P 3596f9392f8746a09722d927914e3f20 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:33:13.129565 13816 tablet_replica.cc:330] T c36b772e33454c9aaf039ceebfc1177c P 99ba8d46c261419aa0813e9e10dc6cd1: stopping tablet replica
I20260807 18:33:13.129843 13816 raft_consensus.cc:2229] T c36b772e33454c9aaf039ceebfc1177c P 99ba8d46c261419aa0813e9e10dc6cd1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:33:13.130043 13816 raft_consensus.cc:2258] T c36b772e33454c9aaf039ceebfc1177c P 99ba8d46c261419aa0813e9e10dc6cd1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:33:13.130383 13818 ts_tablet_manager.cc:1905] T c36b772e33454c9aaf039ceebfc1177c P 97e3153b9b104e5a9fca75dcbb61571b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:33:13.131289 13817 ts_tablet_manager.cc:1905] T c36b772e33454c9aaf039ceebfc1177c P 3596f9392f8746a09722d927914e3f20: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:33:13.133733 13818 ts_tablet_manager.cc:1918] T c36b772e33454c9aaf039ceebfc1177c P 97e3153b9b104e5a9fca75dcbb61571b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 18:33:13.133844 13818 log.cc:1196] T c36b772e33454c9aaf039ceebfc1177c P 97e3153b9b104e5a9fca75dcbb61571b: Deleting WAL directory at /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127579900523-11321-0/minicluster-data/ts-0/wal/wals/c36b772e33454c9aaf039ceebfc1177c
I20260807 18:33:13.134171 13818 ts_tablet_manager.cc:1939] T c36b772e33454c9aaf039ceebfc1177c P 97e3153b9b104e5a9fca75dcbb61571b: Deleting consensus metadata
I20260807 18:33:13.134600 13817 ts_tablet_manager.cc:1918] T c36b772e33454c9aaf039ceebfc1177c P 3596f9392f8746a09722d927914e3f20: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 18:33:13.134696 13817 log.cc:1196] T c36b772e33454c9aaf039ceebfc1177c P 3596f9392f8746a09722d927914e3f20: Deleting WAL directory at /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127579900523-11321-0/minicluster-data/ts-2/wal/wals/c36b772e33454c9aaf039ceebfc1177c
I20260807 18:33:13.135042 13817 ts_tablet_manager.cc:1939] T c36b772e33454c9aaf039ceebfc1177c P 3596f9392f8746a09722d927914e3f20: Deleting consensus metadata
I20260807 18:33:13.135639 13515 tablet_service.cc:1467] Processing CreateTablet for tablet 5edf6bbf50684b35b1d4c863dbc7aae3 (DEFAULT_TABLE table=test-table [id=160f122718f247bf9914afd5f22f45fe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:33:13.135965 13515 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5edf6bbf50684b35b1d4c863dbc7aae3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:33:13.135975 13206 catalog_manager.cc:4841] TS 97e3153b9b104e5a9fca75dcbb61571b (127.11.14.65:45267): tablet c36b772e33454c9aaf039ceebfc1177c (table test-table [id=f3b352ba6e6947c18b4185308dae76bc]) successfully deleted
I20260807 18:33:13.136610 13371 tablet_service.cc:1467] Processing CreateTablet for tablet 5edf6bbf50684b35b1d4c863dbc7aae3 (DEFAULT_TABLE table=test-table [id=160f122718f247bf9914afd5f22f45fe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:33:13.136875 13371 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5edf6bbf50684b35b1d4c863dbc7aae3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:33:13.137089 13205 catalog_manager.cc:4841] TS 3596f9392f8746a09722d927914e3f20 (127.11.14.67:41295): tablet c36b772e33454c9aaf039ceebfc1177c (table test-table [id=f3b352ba6e6947c18b4185308dae76bc]) successfully deleted
I20260807 18:33:13.138703 13771 tablet_bootstrap.cc:492] T 5edf6bbf50684b35b1d4c863dbc7aae3 P 99ba8d46c261419aa0813e9e10dc6cd1: Bootstrap starting.
I20260807 18:33:13.138876 13766 tablet_bootstrap.cc:492] T 5edf6bbf50684b35b1d4c863dbc7aae3 P 97e3153b9b104e5a9fca75dcbb61571b: Bootstrap starting.
I20260807 18:33:13.139287 13672 tablet_service.cc:1467] Processing CreateTablet for tablet 5edf6bbf50684b35b1d4c863dbc7aae3 (DEFAULT_TABLE table=test-table [id=160f122718f247bf9914afd5f22f45fe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:33:13.139588 13672 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5edf6bbf50684b35b1d4c863dbc7aae3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:33:13.139770 13771 tablet_bootstrap.cc:654] T 5edf6bbf50684b35b1d4c863dbc7aae3 P 99ba8d46c261419aa0813e9e10dc6cd1: Neither blocks nor log segments found. Creating new log.
I20260807 18:33:13.139936 13766 tablet_bootstrap.cc:654] T 5edf6bbf50684b35b1d4c863dbc7aae3 P 97e3153b9b104e5a9fca75dcbb61571b: Neither blocks nor log segments found. Creating new log.
I20260807 18:33:13.141335 13766 tablet_bootstrap.cc:492] T 5edf6bbf50684b35b1d4c863dbc7aae3 P 97e3153b9b104e5a9fca75dcbb61571b: No bootstrap required, opened a new log
I20260807 18:33:13.141424 13766 ts_tablet_manager.cc:1397] T 5edf6bbf50684b35b1d4c863dbc7aae3 P 97e3153b9b104e5a9fca75dcbb61571b: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260807 18:33:13.141467 13771 tablet_bootstrap.cc:492] T 5edf6bbf50684b35b1d4c863dbc7aae3 P 99ba8d46c261419aa0813e9e10dc6cd1: No bootstrap required, opened a new log
I20260807 18:33:13.141546 13771 ts_tablet_manager.cc:1397] T 5edf6bbf50684b35b1d4c863dbc7aae3 P 99ba8d46c261419aa0813e9e10dc6cd1: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260807 18:33:13.141891 13766 raft_consensus.cc:348] T 5edf6bbf50684b35b1d4c863dbc7aae3 P 97e3153b9b104e5a9fca75dcbb61571b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97e3153b9b104e5a9fca75dcbb61571b" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 45267 } } peers { permanent_uuid: "99ba8d46c261419aa0813e9e10dc6cd1" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 45171 } } peers { permanent_uuid: "3596f9392f8746a09722d927914e3f20" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 41295 } }
I20260807 18:33:13.142017 13766 raft_consensus.cc:374] T 5edf6bbf50684b35b1d4c863dbc7aae3 P 97e3153b9b104e5a9fca75dcbb61571b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:33:13.142052 13766 raft_consensus.cc:729] T 5edf6bbf50684b35b1d4c863dbc7aae3 P 97e3153b9b104e5a9fca75dcbb61571b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97e3153b9b104e5a9fca75dcbb61571b, State: Initialized, Role: FOLLOWER
I20260807 18:33:13.141988 13771 raft_consensus.cc:348] T 5edf6bbf50684b35b1d4c863dbc7aae3 P 99ba8d46c261419aa0813e9e10dc6cd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97e3153b9b104e5a9fca75dcbb61571b" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 45267 } } peers { permanent_uuid: "99ba8d46c261419aa0813e9e10dc6cd1" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 45171 } } peers { permanent_uuid: "3596f9392f8746a09722d927914e3f20" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 41295 } }
I20260807 18:33:13.142102 13771 raft_consensus.cc:374] T 5edf6bbf50684b35b1d4c863dbc7aae3 P 99ba8d46c261419aa0813e9e10dc6cd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:33:13.142132 13771 raft_consensus.cc:729] T 5edf6bbf50684b35b1d4c863dbc7aae3 P 99ba8d46c261419aa0813e9e10dc6cd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99ba8d46c261419aa0813e9e10dc6cd1, State: Initialized, Role: FOLLOWER
I20260807 18:33:13.142175 13766 consensus_queue.cc:260] T 5edf6bbf50684b35b1d4c863dbc7aae3 P 97e3153b9b104e5a9fca75dcbb61571b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97e3153b9b104e5a9fca75dcbb61571b" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 45267 } } peers { permanent_uuid: "99ba8d46c261419aa0813e9e10dc6cd1" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 45171 } } peers { permanent_uuid: "3596f9392f8746a09722d927914e3f20" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 41295 } }
I20260807 18:33:13.142248 13771 consensus_queue.cc:260] T 5edf6bbf50684b35b1d4c863dbc7aae3 P 99ba8d46c261419aa0813e9e10dc6cd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97e3153b9b104e5a9fca75dcbb61571b" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 45267 } } peers { permanent_uuid: "99ba8d46c261419aa0813e9e10dc6cd1" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 45171 } } peers { permanent_uuid: "3596f9392f8746a09722d927914e3f20" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 41295 } }
I20260807 18:33:13.142454 13766 ts_tablet_manager.cc:1428] T 5edf6bbf50684b35b1d4c863dbc7aae3 P 97e3153b9b104e5a9fca75dcbb61571b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 18:33:13.142513 13771 ts_tablet_manager.cc:1428] T 5edf6bbf50684b35b1d4c863dbc7aae3 P 99ba8d46c261419aa0813e9e10dc6cd1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 18:33:13.142727 13767 tablet_bootstrap.cc:492] T 5edf6bbf50684b35b1d4c863dbc7aae3 P 3596f9392f8746a09722d927914e3f20: Bootstrap starting.
I20260807 18:33:13.143738 13767 tablet_bootstrap.cc:654] T 5edf6bbf50684b35b1d4c863dbc7aae3 P 3596f9392f8746a09722d927914e3f20: Neither blocks nor log segments found. Creating new log.
I20260807 18:33:13.144917 13767 tablet_bootstrap.cc:492] T 5edf6bbf50684b35b1d4c863dbc7aae3 P 3596f9392f8746a09722d927914e3f20: No bootstrap required, opened a new log
I20260807 18:33:13.145008 13767 ts_tablet_manager.cc:1397] T 5edf6bbf50684b35b1d4c863dbc7aae3 P 3596f9392f8746a09722d927914e3f20: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260807 18:33:13.145406 13767 raft_consensus.cc:348] T 5edf6bbf50684b35b1d4c863dbc7aae3 P 3596f9392f8746a09722d927914e3f20 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97e3153b9b104e5a9fca75dcbb61571b" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 45267 } } peers { permanent_uuid: "99ba8d46c261419aa0813e9e10dc6cd1" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 45171 } } peers { permanent_uuid: "3596f9392f8746a09722d927914e3f20" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 41295 } }
I20260807 18:33:13.145511 13767 raft_consensus.cc:374] T 5edf6bbf50684b35b1d4c863dbc7aae3 P 3596f9392f8746a09722d927914e3f20 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:33:13.145540 13767 raft_consensus.cc:729] T 5edf6bbf50684b35b1d4c863dbc7aae3 P 3596f9392f8746a09722d927914e3f20 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3596f9392f8746a09722d927914e3f20, State: Initialized, Role: FOLLOWER
I20260807 18:33:13.145639 13816 ts_tablet_manager.cc:1905] T c36b772e33454c9aaf039ceebfc1177c P 99ba8d46c261419aa0813e9e10dc6cd1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:33:13.145663 13767 consensus_queue.cc:260] T 5edf6bbf50684b35b1d4c863dbc7aae3 P 3596f9392f8746a09722d927914e3f20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97e3153b9b104e5a9fca75dcbb61571b" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 45267 } } peers { permanent_uuid: "99ba8d46c261419aa0813e9e10dc6cd1" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 45171 } } peers { permanent_uuid: "3596f9392f8746a09722d927914e3f20" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 41295 } }
I20260807 18:33:13.145926 13767 ts_tablet_manager.cc:1428] T 5edf6bbf50684b35b1d4c863dbc7aae3 P 3596f9392f8746a09722d927914e3f20: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 18:33:13.148588 13816 ts_tablet_manager.cc:1918] T c36b772e33454c9aaf039ceebfc1177c P 99ba8d46c261419aa0813e9e10dc6cd1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 18:33:13.148705 13816 log.cc:1196] T c36b772e33454c9aaf039ceebfc1177c P 99ba8d46c261419aa0813e9e10dc6cd1: Deleting WAL directory at /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127579900523-11321-0/minicluster-data/ts-1/wal/wals/c36b772e33454c9aaf039ceebfc1177c
I20260807 18:33:13.149060 13816 ts_tablet_manager.cc:1939] T c36b772e33454c9aaf039ceebfc1177c P 99ba8d46c261419aa0813e9e10dc6cd1: Deleting consensus metadata
I20260807 18:33:13.150373 13207 catalog_manager.cc:4841] TS 99ba8d46c261419aa0813e9e10dc6cd1 (127.11.14.66:45171): tablet c36b772e33454c9aaf039ceebfc1177c (table test-table [id=f3b352ba6e6947c18b4185308dae76bc]) successfully deleted
I20260807 18:33:13.217141 13772 raft_consensus.cc:482] T 5edf6bbf50684b35b1d4c863dbc7aae3 P 97e3153b9b104e5a9fca75dcbb61571b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:33:13.217303 13772 raft_consensus.cc:504] T 5edf6bbf50684b35b1d4c863dbc7aae3 P 97e3153b9b104e5a9fca75dcbb61571b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97e3153b9b104e5a9fca75dcbb61571b" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 45267 } } peers { permanent_uuid: "99ba8d46c261419aa0813e9e10dc6cd1" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 45171 } } peers { permanent_uuid: "3596f9392f8746a09722d927914e3f20" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 41295 } }
I20260807 18:33:13.218825 13772 leader_election.cc:288] T 5edf6bbf50684b35b1d4c863dbc7aae3 P 97e3153b9b104e5a9fca75dcbb61571b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99ba8d46c261419aa0813e9e10dc6cd1 (127.11.14.66:45171), 3596f9392f8746a09722d927914e3f20 (127.11.14.67:41295)
I20260807 18:33:13.223606 13822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:13.219084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.67:41295 (local address 127.11.14.65:60121)
0807 18:33:13.219895 (+ 811us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:13.219910 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:13.220013 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:13.220365 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:13.220368 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:13.220381 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:33:13.220480 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:13.220489 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:13.221284 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:13.221288 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:13.222386 (+ 1098us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:13.222397 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:13.223112 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:13.223116 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:13.223237 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:13.223257 (+ 20us) client_negotiation.cc:769] Sending connection context
0807 18:33:13.223347 (+ 90us) client_negotiation.cc:240] Negotiation successful
0807 18:33:13.223423 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":741,"thread_start_us":127,"threads_started":1}
I20260807 18:33:13.223598 13803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:13.219462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.65:60121 (local address 127.11.14.67:41295)
0807 18:33:13.219623 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:13.219628 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:13.219995 (+ 367us) server_negotiation.cc:407] Connection header received
0807 18:33:13.220102 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:13.220106 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:13.220157 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:13.220237 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:33:13.220584 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:13.221165 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:13.222556 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:13.222958 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:13.223104 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:13.223169 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:13.223384 (+ 215us) server_negotiation.cc:299] Negotiation successful
0807 18:33:13.223466 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260807 18:33:13.224228 13695 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5edf6bbf50684b35b1d4c863dbc7aae3" candidate_uuid: "97e3153b9b104e5a9fca75dcbb61571b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3596f9392f8746a09722d927914e3f20" is_pre_election: true
I20260807 18:33:13.224525 13695 raft_consensus.cc:2454] T 5edf6bbf50684b35b1d4c863dbc7aae3 P 3596f9392f8746a09722d927914e3f20 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 97e3153b9b104e5a9fca75dcbb61571b in term 0.
I20260807 18:33:13.224589 13823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:13.219056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.66:45171 (local address 127.11.14.65:34873)
0807 18:33:13.219610 (+ 554us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:13.219628 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:13.219747 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:13.220236 (+ 489us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:13.220239 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:13.220255 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:33:13.220395 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:13.220424 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:13.221658 (+ 1234us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:13.221662 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:13.222872 (+ 1210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:13.222884 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:13.224147 (+ 1263us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:13.224150 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:13.224256 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:13.224282 (+ 26us) client_negotiation.cc:769] Sending connection context
0807 18:33:13.224378 (+ 96us) client_negotiation.cc:240] Negotiation successful
0807 18:33:13.224444 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":447,"thread_start_us":111,"threads_started":1}
I20260807 18:33:13.224911 13824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:13.219255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.65:34873 (local address 127.11.14.66:45171)
0807 18:33:13.219992 (+ 737us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:13.219997 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:13.220011 (+ 14us) server_negotiation.cc:407] Connection header received
0807 18:33:13.220071 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:13.220074 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:13.220121 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:13.220228 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:33:13.220531 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:13.221485 (+ 954us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:13.223028 (+ 1543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:13.224020 (+ 992us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:13.224163 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:13.224214 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:13.224483 (+ 269us) server_negotiation.cc:299] Negotiation successful
0807 18:33:13.224565 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":640,"thread_start_us":81,"threads_started":1}
I20260807 18:33:13.225046 13317 leader_election.cc:302] T 5edf6bbf50684b35b1d4c863dbc7aae3 P 97e3153b9b104e5a9fca75dcbb61571b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3596f9392f8746a09722d927914e3f20, 97e3153b9b104e5a9fca75dcbb61571b; no voters:
I20260807 18:33:13.225275 13541 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5edf6bbf50684b35b1d4c863dbc7aae3" candidate_uuid: "97e3153b9b104e5a9fca75dcbb61571b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99ba8d46c261419aa0813e9e10dc6cd1" is_pre_election: true
I20260807 18:33:13.225360 13772 raft_consensus.cc:2781] T 5edf6bbf50684b35b1d4c863dbc7aae3 P 97e3153b9b104e5a9fca75dcbb61571b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:33:13.225416 13541 raft_consensus.cc:2454] T 5edf6bbf50684b35b1d4c863dbc7aae3 P 99ba8d46c261419aa0813e9e10dc6cd1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 97e3153b9b104e5a9fca75dcbb61571b in term 0.
I20260807 18:33:13.225430 13772 raft_consensus.cc:482] T 5edf6bbf50684b35b1d4c863dbc7aae3 P 97e3153b9b104e5a9fca75dcbb61571b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:33:13.225483 13772 raft_consensus.cc:3037] T 5edf6bbf50684b35b1d4c863dbc7aae3 P 97e3153b9b104e5a9fca75dcbb61571b [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:33:13.226557 13772 raft_consensus.cc:504] T 5edf6bbf50684b35b1d4c863dbc7aae3 P 97e3153b9b104e5a9fca75dcbb61571b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97e3153b9b104e5a9fca75dcbb61571b" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 45267 } } peers { permanent_uuid: "99ba8d46c261419aa0813e9e10dc6cd1" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 45171 } } peers { permanent_uuid: "3596f9392f8746a09722d927914e3f20" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 41295 } }
I20260807 18:33:13.226984 13772 leader_election.cc:288] T 5edf6bbf50684b35b1d4c863dbc7aae3 P 97e3153b9b104e5a9fca75dcbb61571b [CANDIDATE]: Term 1 election: Requested vote from peers 99ba8d46c261419aa0813e9e10dc6cd1 (127.11.14.66:45171), 3596f9392f8746a09722d927914e3f20 (127.11.14.67:41295)
I20260807 18:33:13.227346 13695 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5edf6bbf50684b35b1d4c863dbc7aae3" candidate_uuid: "97e3153b9b104e5a9fca75dcbb61571b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3596f9392f8746a09722d927914e3f20"
I20260807 18:33:13.227495 13695 raft_consensus.cc:3037] T 5edf6bbf50684b35b1d4c863dbc7aae3 P 3596f9392f8746a09722d927914e3f20 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:33:13.227777 13541 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5edf6bbf50684b35b1d4c863dbc7aae3" candidate_uuid: "97e3153b9b104e5a9fca75dcbb61571b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99ba8d46c261419aa0813e9e10dc6cd1"
I20260807 18:33:13.227885 13541 raft_consensus.cc:3037] T 5edf6bbf50684b35b1d4c863dbc7aae3 P 99ba8d46c261419aa0813e9e10dc6cd1 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:33:13.228528 13695 raft_consensus.cc:2454] T 5edf6bbf50684b35b1d4c863dbc7aae3 P 3596f9392f8746a09722d927914e3f20 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 97e3153b9b104e5a9fca75dcbb61571b in term 1.
I20260807 18:33:13.228859 13317 leader_election.cc:302] T 5edf6bbf50684b35b1d4c863dbc7aae3 P 97e3153b9b104e5a9fca75dcbb61571b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3596f9392f8746a09722d927914e3f20, 97e3153b9b104e5a9fca75dcbb61571b; no voters:
I20260807 18:33:13.228989 13541 raft_consensus.cc:2454] T 5edf6bbf50684b35b1d4c863dbc7aae3 P 99ba8d46c261419aa0813e9e10dc6cd1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 97e3153b9b104e5a9fca75dcbb61571b in term 1.
I20260807 18:33:13.229055 13772 raft_consensus.cc:2781] T 5edf6bbf50684b35b1d4c863dbc7aae3 P 97e3153b9b104e5a9fca75dcbb61571b [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:33:13.229301 13772 raft_consensus.cc:686] T 5edf6bbf50684b35b1d4c863dbc7aae3 P 97e3153b9b104e5a9fca75dcbb61571b [term 1 LEADER]: Becoming Leader. State: Replica: 97e3153b9b104e5a9fca75dcbb61571b, State: Running, Role: LEADER
I20260807 18:33:13.229590 13772 consensus_queue.cc:237] T 5edf6bbf50684b35b1d4c863dbc7aae3 P 97e3153b9b104e5a9fca75dcbb61571b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97e3153b9b104e5a9fca75dcbb61571b" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 45267 } } peers { permanent_uuid: "99ba8d46c261419aa0813e9e10dc6cd1" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 45171 } } peers { permanent_uuid: "3596f9392f8746a09722d927914e3f20" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 41295 } }
I20260807 18:33:13.232118 13221 catalog_manager.cc:5495] T 5edf6bbf50684b35b1d4c863dbc7aae3 P 97e3153b9b104e5a9fca75dcbb61571b reported cstate change: term changed from 0 to 1, leader changed from <none> to 97e3153b9b104e5a9fca75dcbb61571b (127.11.14.65). New cstate: current_term: 1 leader_uuid: "97e3153b9b104e5a9fca75dcbb61571b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97e3153b9b104e5a9fca75dcbb61571b" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 45267 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "99ba8d46c261419aa0813e9e10dc6cd1" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 45171 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3596f9392f8746a09722d927914e3f20" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 41295 } health_report { overall_health: UNKNOWN } } }
I20260807 18:33:13.261901 13828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:13.257668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36726 (local address 127.11.14.65:45267)
0807 18:33:13.257985 (+ 317us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:13.257992 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:13.258004 (+ 12us) server_negotiation.cc:407] Connection header received
0807 18:33:13.258078 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:13.258080 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:13.258134 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:13.258253 (+ 119us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:33:13.258537 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:13.259142 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:13.260370 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:13.260598 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:13.260680 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:13.260967 (+ 287us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:33:13.261445 (+ 478us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:33:13.261612 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:13.261682 (+ 70us) server_negotiation.cc:299] Negotiation successful
0807 18:33:13.261738 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":243,"thread_start_us":148,"threads_started":1}
I20260807 18:33:13.277168 13803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:13.272383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.65:50733 (local address 127.11.14.67:41295)
0807 18:33:13.272505 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:13.272510 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:13.272525 (+ 15us) server_negotiation.cc:407] Connection header received
0807 18:33:13.272619 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:13.272622 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:13.272684 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:13.272756 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:33:13.273111 (+ 355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:13.273934 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:13.275928 (+ 1994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:13.276268 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:13.276378 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:13.276705 (+ 327us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:33:13.276791 (+ 86us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:33:13.276864 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:13.276961 (+ 97us) server_negotiation.cc:299] Negotiation successful
0807 18:33:13.277021 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260807 18:33:13.277218 13832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:13.272063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.67:41295 (local address 127.11.14.65:50733)
0807 18:33:13.272402 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:13.272417 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:13.272515 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:13.272783 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:13.272786 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:13.272794 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 18:33:13.272939 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:13.272945 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:13.274044 (+ 1099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:13.274048 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:13.275785 (+ 1737us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:13.275798 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:13.276392 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:13.276396 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:13.276492 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:13.276549 (+ 57us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 18:33:13.276882 (+ 333us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 18:33:13.276888 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 18:33:13.276950 (+ 62us) client_negotiation.cc:240] Negotiation successful
0807 18:33:13.277033 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":112,"threads_started":1}
I20260807 18:33:13.284446 13695 raft_consensus.cc:1261] T 5edf6bbf50684b35b1d4c863dbc7aae3 P 3596f9392f8746a09722d927914e3f20 [term 1 FOLLOWER]: Refusing update from remote peer 97e3153b9b104e5a9fca75dcbb61571b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:33:13.284490 13540 raft_consensus.cc:1261] T 5edf6bbf50684b35b1d4c863dbc7aae3 P 99ba8d46c261419aa0813e9e10dc6cd1 [term 1 FOLLOWER]: Refusing update from remote peer 97e3153b9b104e5a9fca75dcbb61571b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:33:13.285009 13825 consensus_queue.cc:1035] T 5edf6bbf50684b35b1d4c863dbc7aae3 P 97e3153b9b104e5a9fca75dcbb61571b [LEADER]: Connected to new peer: Peer: permanent_uuid: "99ba8d46c261419aa0813e9e10dc6cd1" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 45171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:33:13.285202 13772 consensus_queue.cc:1035] T 5edf6bbf50684b35b1d4c863dbc7aae3 P 97e3153b9b104e5a9fca75dcbb61571b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3596f9392f8746a09722d927914e3f20" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 41295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:33:13.306058 13814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:13.302235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.65:45267 (local address 127.11.14.67:41003)
0807 18:33:13.302407 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:13.302422 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:13.302519 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:13.302861 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:13.302864 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:13.302872 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 18:33:13.303007 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:13.303015 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:13.303956 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:13.303960 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:13.304616 (+ 656us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:13.304626 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:13.305203 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:13.305210 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:13.305297 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:13.305333 (+ 36us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 18:33:13.305769 (+ 436us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 18:33:13.305779 (+ 10us) client_negotiation.cc:769] Sending connection context
0807 18:33:13.305875 (+ 96us) client_negotiation.cc:240] Negotiation successful
0807 18:33:13.305949 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":80}
I20260807 18:33:13.306094 13828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:13.302291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.67:41003 (local address 127.11.14.65:45267)
0807 18:33:13.302399 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:13.302403 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:13.302487 (+ 84us) server_negotiation.cc:407] Connection header received
0807 18:33:13.302605 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:13.302608 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:13.302663 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:13.302735 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:33:13.303139 (+ 404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:13.303837 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:13.304743 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:13.305095 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:13.305200 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:13.305464 (+ 264us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:33:13.305552 (+ 88us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:33:13.305628 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:13.305905 (+ 277us) server_negotiation.cc:299] Negotiation successful
0807 18:33:13.305966 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
W20260807 18:33:13.306836 13795 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260807 18:33:14.872665 13221 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54826:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:33:14.872895 13221 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54826:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:33:14.875613 13221 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e09adbeddf294788bec26d0474bcf37c: Sending DeleteTablet for 3 replicas of tablet 5edf6bbf50684b35b1d4c863dbc7aae3
I20260807 18:33:14.876200 13371 tablet_service.cc:1514] Processing DeleteTablet for tablet 5edf6bbf50684b35b1d4c863dbc7aae3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:33:14 UTC) from {username='kudu', principal='kudu/127.11.14.126@KRBTEST.COM'} at 127.0.0.1:36712
I20260807 18:33:14.876253 13515 tablet_service.cc:1514] Processing DeleteTablet for tablet 5edf6bbf50684b35b1d4c863dbc7aae3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:33:14 UTC) from {username='kudu', principal='kudu/127.11.14.126@KRBTEST.COM'} at 127.0.0.1:55458
I20260807 18:33:14.876410 13672 tablet_service.cc:1514] Processing DeleteTablet for tablet 5edf6bbf50684b35b1d4c863dbc7aae3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:33:14 UTC) from {username='kudu', principal='kudu/127.11.14.126@KRBTEST.COM'} at 127.0.0.1:50110
I20260807 18:33:14.876633 13862 tablet_replica.cc:330] T 5edf6bbf50684b35b1d4c863dbc7aae3 P 97e3153b9b104e5a9fca75dcbb61571b: stopping tablet replica
I20260807 18:33:14.876796 13862 raft_consensus.cc:2229] T 5edf6bbf50684b35b1d4c863dbc7aae3 P 97e3153b9b104e5a9fca75dcbb61571b [term 1 LEADER]: Raft consensus shutting down.
I20260807 18:33:14.876896 13863 tablet_replica.cc:330] T 5edf6bbf50684b35b1d4c863dbc7aae3 P 3596f9392f8746a09722d927914e3f20: stopping tablet replica
I20260807 18:33:14.877031 13863 raft_consensus.cc:2229] T 5edf6bbf50684b35b1d4c863dbc7aae3 P 3596f9392f8746a09722d927914e3f20 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:33:14.877166 13863 raft_consensus.cc:2258] T 5edf6bbf50684b35b1d4c863dbc7aae3 P 3596f9392f8746a09722d927914e3f20 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:33:14.877339 13862 raft_consensus.cc:2258] T 5edf6bbf50684b35b1d4c863dbc7aae3 P 97e3153b9b104e5a9fca75dcbb61571b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:33:14.877949 13862 ts_tablet_manager.cc:1905] T 5edf6bbf50684b35b1d4c863dbc7aae3 P 97e3153b9b104e5a9fca75dcbb61571b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:33:14.877975 13863 ts_tablet_manager.cc:1905] T 5edf6bbf50684b35b1d4c863dbc7aae3 P 3596f9392f8746a09722d927914e3f20: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:33:14.880697 13860 tablet_replica.cc:330] T 5edf6bbf50684b35b1d4c863dbc7aae3 P 99ba8d46c261419aa0813e9e10dc6cd1: stopping tablet replica
I20260807 18:33:14.880872 13860 raft_consensus.cc:2229] T 5edf6bbf50684b35b1d4c863dbc7aae3 P 99ba8d46c261419aa0813e9e10dc6cd1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:33:14.880937 13862 ts_tablet_manager.cc:1918] T 5edf6bbf50684b35b1d4c863dbc7aae3 P 97e3153b9b104e5a9fca75dcbb61571b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 18:33:14.881012 13862 log.cc:1196] T 5edf6bbf50684b35b1d4c863dbc7aae3 P 97e3153b9b104e5a9fca75dcbb61571b: Deleting WAL directory at /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127579900523-11321-0/minicluster-data/ts-0/wal/wals/5edf6bbf50684b35b1d4c863dbc7aae3
I20260807 18:33:14.881029 13860 raft_consensus.cc:2258] T 5edf6bbf50684b35b1d4c863dbc7aae3 P 99ba8d46c261419aa0813e9e10dc6cd1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:33:14.881332 13862 ts_tablet_manager.cc:1939] T 5edf6bbf50684b35b1d4c863dbc7aae3 P 97e3153b9b104e5a9fca75dcbb61571b: Deleting consensus metadata
I20260807 18:33:14.881474 13863 ts_tablet_manager.cc:1918] T 5edf6bbf50684b35b1d4c863dbc7aae3 P 3596f9392f8746a09722d927914e3f20: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 18:33:14.881561 13863 log.cc:1196] T 5edf6bbf50684b35b1d4c863dbc7aae3 P 3596f9392f8746a09722d927914e3f20: Deleting WAL directory at /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127579900523-11321-0/minicluster-data/ts-2/wal/wals/5edf6bbf50684b35b1d4c863dbc7aae3
I20260807 18:33:14.881824 13863 ts_tablet_manager.cc:1939] T 5edf6bbf50684b35b1d4c863dbc7aae3 P 3596f9392f8746a09722d927914e3f20: Deleting consensus metadata
I20260807 18:33:14.882326 13206 catalog_manager.cc:4841] TS 97e3153b9b104e5a9fca75dcbb61571b (127.11.14.65:45267): tablet 5edf6bbf50684b35b1d4c863dbc7aae3 (table test-table [id=160f122718f247bf9914afd5f22f45fe]) successfully deleted
I20260807 18:33:14.882704 13205 catalog_manager.cc:4841] TS 3596f9392f8746a09722d927914e3f20 (127.11.14.67:41295): tablet 5edf6bbf50684b35b1d4c863dbc7aae3 (table test-table [id=160f122718f247bf9914afd5f22f45fe]) successfully deleted
I20260807 18:33:14.884006 13860 ts_tablet_manager.cc:1905] T 5edf6bbf50684b35b1d4c863dbc7aae3 P 99ba8d46c261419aa0813e9e10dc6cd1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:33:14.886852 13860 ts_tablet_manager.cc:1918] T 5edf6bbf50684b35b1d4c863dbc7aae3 P 99ba8d46c261419aa0813e9e10dc6cd1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 18:33:14.886950 13860 log.cc:1196] T 5edf6bbf50684b35b1d4c863dbc7aae3 P 99ba8d46c261419aa0813e9e10dc6cd1: Deleting WAL directory at /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127579900523-11321-0/minicluster-data/ts-1/wal/wals/5edf6bbf50684b35b1d4c863dbc7aae3
I20260807 18:33:14.887199 13860 ts_tablet_manager.cc:1939] T 5edf6bbf50684b35b1d4c863dbc7aae3 P 99ba8d46c261419aa0813e9e10dc6cd1: Deleting consensus metadata
I20260807 18:33:14.887970 13207 catalog_manager.cc:4841] TS 99ba8d46c261419aa0813e9e10dc6cd1 (127.11.14.66:45171): tablet 5edf6bbf50684b35b1d4c863dbc7aae3 (table test-table [id=160f122718f247bf9914afd5f22f45fe]) successfully deleted
Aug 07 18:33:14 dist-test-slave-6k22 krb5kdc[13174](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127594, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 07 18:33:14 dist-test-slave-6k22 krb5kdc[13174](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127594, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.11.14.126@KRBTEST.COM
I20260807 18:33:14.906950 13871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:14.897721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54842 (local address 127.11.14.126:36947)
0807 18:33:14.898094 (+ 373us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:14.898103 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:14.898119 (+ 16us) server_negotiation.cc:407] Connection header received
0807 18:33:14.898273 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:14.898275 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:14.898325 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:14.898438 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:33:14.899268 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:14.899894 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:14.900758 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:14.901001 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:14.901114 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:14.903650 (+ 2536us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:33:14.903654 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:33:14.903656 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:33:14.903681 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:33:14.905144 (+ 1463us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:14.905585 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:14.905590 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:14.905591 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:14.905682 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:14.905980 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:14.905983 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:14.905987 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:14.906205 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:33:14.906352 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:14.906616 (+ 264us) server_negotiation.cc:299] Negotiation successful
0807 18:33:14.906752 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":283,"thread_start_us":189,"threads_started":1}
I20260807 18:33:14.909811 13221 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54842:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 18:33:14.910231 13221 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 18:33:14.915041 13371 tablet_service.cc:1467] Processing CreateTablet for tablet 9415b291276b458380f53455ce4cd92e (DEFAULT_TABLE table=test-table [id=44af5ca7d92447a6a55ae7ea2d5125e2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:33:14.915033 13515 tablet_service.cc:1467] Processing CreateTablet for tablet 9415b291276b458380f53455ce4cd92e (DEFAULT_TABLE table=test-table [id=44af5ca7d92447a6a55ae7ea2d5125e2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:33:14.915097 13672 tablet_service.cc:1467] Processing CreateTablet for tablet 9415b291276b458380f53455ce4cd92e (DEFAULT_TABLE table=test-table [id=44af5ca7d92447a6a55ae7ea2d5125e2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:33:14.915383 13515 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9415b291276b458380f53455ce4cd92e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:33:14.915380 13371 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9415b291276b458380f53455ce4cd92e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:33:14.915380 13672 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9415b291276b458380f53455ce4cd92e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:33:14.917878 13872 tablet_bootstrap.cc:492] T 9415b291276b458380f53455ce4cd92e P 99ba8d46c261419aa0813e9e10dc6cd1: Bootstrap starting.
I20260807 18:33:14.918617 13873 tablet_bootstrap.cc:492] T 9415b291276b458380f53455ce4cd92e P 97e3153b9b104e5a9fca75dcbb61571b: Bootstrap starting.
I20260807 18:33:14.918999 13872 tablet_bootstrap.cc:654] T 9415b291276b458380f53455ce4cd92e P 99ba8d46c261419aa0813e9e10dc6cd1: Neither blocks nor log segments found. Creating new log.
I20260807 18:33:14.919714 13873 tablet_bootstrap.cc:654] T 9415b291276b458380f53455ce4cd92e P 97e3153b9b104e5a9fca75dcbb61571b: Neither blocks nor log segments found. Creating new log.
I20260807 18:33:14.919762 13874 tablet_bootstrap.cc:492] T 9415b291276b458380f53455ce4cd92e P 3596f9392f8746a09722d927914e3f20: Bootstrap starting.
I20260807 18:33:14.920770 13874 tablet_bootstrap.cc:654] T 9415b291276b458380f53455ce4cd92e P 3596f9392f8746a09722d927914e3f20: Neither blocks nor log segments found. Creating new log.
I20260807 18:33:14.921025 13873 tablet_bootstrap.cc:492] T 9415b291276b458380f53455ce4cd92e P 97e3153b9b104e5a9fca75dcbb61571b: No bootstrap required, opened a new log
I20260807 18:33:14.921132 13873 ts_tablet_manager.cc:1397] T 9415b291276b458380f53455ce4cd92e P 97e3153b9b104e5a9fca75dcbb61571b: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260807 18:33:14.921583 13873 raft_consensus.cc:348] T 9415b291276b458380f53455ce4cd92e P 97e3153b9b104e5a9fca75dcbb61571b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97e3153b9b104e5a9fca75dcbb61571b" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 45267 } } peers { permanent_uuid: "99ba8d46c261419aa0813e9e10dc6cd1" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 45171 } } peers { permanent_uuid: "3596f9392f8746a09722d927914e3f20" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 41295 } }
I20260807 18:33:14.921674 13872 tablet_bootstrap.cc:492] T 9415b291276b458380f53455ce4cd92e P 99ba8d46c261419aa0813e9e10dc6cd1: No bootstrap required, opened a new log
I20260807 18:33:14.921710 13873 raft_consensus.cc:374] T 9415b291276b458380f53455ce4cd92e P 97e3153b9b104e5a9fca75dcbb61571b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:33:14.921764 13873 raft_consensus.cc:729] T 9415b291276b458380f53455ce4cd92e P 97e3153b9b104e5a9fca75dcbb61571b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97e3153b9b104e5a9fca75dcbb61571b, State: Initialized, Role: FOLLOWER
I20260807 18:33:14.921758 13872 ts_tablet_manager.cc:1397] T 9415b291276b458380f53455ce4cd92e P 99ba8d46c261419aa0813e9e10dc6cd1: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260807 18:33:14.921948 13873 consensus_queue.cc:260] T 9415b291276b458380f53455ce4cd92e P 97e3153b9b104e5a9fca75dcbb61571b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97e3153b9b104e5a9fca75dcbb61571b" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 45267 } } peers { permanent_uuid: "99ba8d46c261419aa0813e9e10dc6cd1" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 45171 } } peers { permanent_uuid: "3596f9392f8746a09722d927914e3f20" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 41295 } }
I20260807 18:33:14.922068 13874 tablet_bootstrap.cc:492] T 9415b291276b458380f53455ce4cd92e P 3596f9392f8746a09722d927914e3f20: No bootstrap required, opened a new log
I20260807 18:33:14.922159 13874 ts_tablet_manager.cc:1397] T 9415b291276b458380f53455ce4cd92e P 3596f9392f8746a09722d927914e3f20: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260807 18:33:14.922224 13872 raft_consensus.cc:348] T 9415b291276b458380f53455ce4cd92e P 99ba8d46c261419aa0813e9e10dc6cd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97e3153b9b104e5a9fca75dcbb61571b" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 45267 } } peers { permanent_uuid: "99ba8d46c261419aa0813e9e10dc6cd1" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 45171 } } peers { permanent_uuid: "3596f9392f8746a09722d927914e3f20" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 41295 } }
I20260807 18:33:14.922324 13873 ts_tablet_manager.cc:1428] T 9415b291276b458380f53455ce4cd92e P 97e3153b9b104e5a9fca75dcbb61571b: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260807 18:33:14.922362 13872 raft_consensus.cc:374] T 9415b291276b458380f53455ce4cd92e P 99ba8d46c261419aa0813e9e10dc6cd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:33:14.922402 13872 raft_consensus.cc:729] T 9415b291276b458380f53455ce4cd92e P 99ba8d46c261419aa0813e9e10dc6cd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99ba8d46c261419aa0813e9e10dc6cd1, State: Initialized, Role: FOLLOWER
I20260807 18:33:14.922526 13872 consensus_queue.cc:260] T 9415b291276b458380f53455ce4cd92e P 99ba8d46c261419aa0813e9e10dc6cd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97e3153b9b104e5a9fca75dcbb61571b" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 45267 } } peers { permanent_uuid: "99ba8d46c261419aa0813e9e10dc6cd1" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 45171 } } peers { permanent_uuid: "3596f9392f8746a09722d927914e3f20" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 41295 } }
I20260807 18:33:14.922571 13874 raft_consensus.cc:348] T 9415b291276b458380f53455ce4cd92e P 3596f9392f8746a09722d927914e3f20 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97e3153b9b104e5a9fca75dcbb61571b" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 45267 } } peers { permanent_uuid: "99ba8d46c261419aa0813e9e10dc6cd1" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 45171 } } peers { permanent_uuid: "3596f9392f8746a09722d927914e3f20" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 41295 } }
I20260807 18:33:14.922695 13874 raft_consensus.cc:374] T 9415b291276b458380f53455ce4cd92e P 3596f9392f8746a09722d927914e3f20 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:33:14.922739 13874 raft_consensus.cc:729] T 9415b291276b458380f53455ce4cd92e P 3596f9392f8746a09722d927914e3f20 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3596f9392f8746a09722d927914e3f20, State: Initialized, Role: FOLLOWER
I20260807 18:33:14.922925 13874 consensus_queue.cc:260] T 9415b291276b458380f53455ce4cd92e P 3596f9392f8746a09722d927914e3f20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97e3153b9b104e5a9fca75dcbb61571b" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 45267 } } peers { permanent_uuid: "99ba8d46c261419aa0813e9e10dc6cd1" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 45171 } } peers { permanent_uuid: "3596f9392f8746a09722d927914e3f20" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 41295 } }
I20260807 18:33:14.922982 13872 ts_tablet_manager.cc:1428] T 9415b291276b458380f53455ce4cd92e P 99ba8d46c261419aa0813e9e10dc6cd1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 18:33:14.923667 13874 ts_tablet_manager.cc:1428] T 9415b291276b458380f53455ce4cd92e P 3596f9392f8746a09722d927914e3f20: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260807 18:33:15.043296 13878 raft_consensus.cc:482] T 9415b291276b458380f53455ce4cd92e P 99ba8d46c261419aa0813e9e10dc6cd1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:33:15.043470 13878 raft_consensus.cc:504] T 9415b291276b458380f53455ce4cd92e P 99ba8d46c261419aa0813e9e10dc6cd1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97e3153b9b104e5a9fca75dcbb61571b" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 45267 } } peers { permanent_uuid: "99ba8d46c261419aa0813e9e10dc6cd1" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 45171 } } peers { permanent_uuid: "3596f9392f8746a09722d927914e3f20" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 41295 } }
I20260807 18:33:15.044697 13878 leader_election.cc:288] T 9415b291276b458380f53455ce4cd92e P 99ba8d46c261419aa0813e9e10dc6cd1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97e3153b9b104e5a9fca75dcbb61571b (127.11.14.65:45267), 3596f9392f8746a09722d927914e3f20 (127.11.14.67:41295)
I20260807 18:33:15.048803 13881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:15.045053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.66:59713 (local address 127.11.14.65:45267)
0807 18:33:15.045370 (+ 317us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:15.045381 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:15.045481 (+ 100us) server_negotiation.cc:407] Connection header received
0807 18:33:15.045655 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:15.045658 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:15.045702 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:15.045799 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:33:15.046114 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:15.046667 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:15.047792 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:15.048191 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:15.048323 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:15.048391 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:15.048562 (+ 171us) server_negotiation.cc:299] Negotiation successful
0807 18:33:15.048639 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":216,"thread_start_us":94,"threads_started":1}
I20260807 18:33:15.048843 13880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:15.044946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.65:45267 (local address 127.11.14.66:59713)
0807 18:33:15.045370 (+ 424us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:15.045385 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:15.045517 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:15.045813 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:15.045816 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:15.045831 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:33:15.045992 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:15.045998 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:15.046785 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:15.046790 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:15.047658 (+ 868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:15.047667 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:15.048319 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:15.048323 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:15.048423 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:15.048446 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 18:33:15.048545 (+ 99us) client_negotiation.cc:240] Negotiation successful
0807 18:33:15.048639 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":361,"mutex_wait_us":25,"thread_start_us":173,"threads_started":1}
I20260807 18:33:15.048846 13879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:15.044967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.67:41295 (local address 127.11.14.66:58849)
0807 18:33:15.045453 (+ 486us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:15.045474 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:15.045601 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:15.045888 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:15.045891 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:15.045907 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:33:15.046062 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:15.046067 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:15.046824 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:15.046828 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:15.047703 (+ 875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:15.047713 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:15.048362 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:15.048366 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:15.048460 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:15.048482 (+ 22us) client_negotiation.cc:769] Sending connection context
0807 18:33:15.048578 (+ 96us) client_negotiation.cc:240] Negotiation successful
0807 18:33:15.048680 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":415,"mutex_wait_us":42,"thread_start_us":154,"threads_started":1}
I20260807 18:33:15.048835 13882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:15.045101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.66:58849 (local address 127.11.14.67:41295)
0807 18:33:15.045407 (+ 306us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:15.045417 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:15.045559 (+ 142us) server_negotiation.cc:407] Connection header received
0807 18:33:15.045733 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:15.045736 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:15.045781 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:15.045877 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:33:15.046184 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:15.046703 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:15.047823 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:15.048223 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:15.048358 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:15.048413 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:15.048597 (+ 184us) server_negotiation.cc:299] Negotiation successful
0807 18:33:15.048689 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":223,"thread_start_us":95,"threads_started":1}
I20260807 18:33:15.049386 13391 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9415b291276b458380f53455ce4cd92e" candidate_uuid: "99ba8d46c261419aa0813e9e10dc6cd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97e3153b9b104e5a9fca75dcbb61571b" is_pre_election: true
I20260807 18:33:15.049584 13391 raft_consensus.cc:2454] T 9415b291276b458380f53455ce4cd92e P 97e3153b9b104e5a9fca75dcbb61571b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99ba8d46c261419aa0813e9e10dc6cd1 in term 0.
I20260807 18:33:15.049693 13695 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9415b291276b458380f53455ce4cd92e" candidate_uuid: "99ba8d46c261419aa0813e9e10dc6cd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3596f9392f8746a09722d927914e3f20" is_pre_election: true
I20260807 18:33:15.049844 13695 raft_consensus.cc:2454] T 9415b291276b458380f53455ce4cd92e P 3596f9392f8746a09722d927914e3f20 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99ba8d46c261419aa0813e9e10dc6cd1 in term 0.
I20260807 18:33:15.050179 13470 leader_election.cc:302] T 9415b291276b458380f53455ce4cd92e P 99ba8d46c261419aa0813e9e10dc6cd1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 97e3153b9b104e5a9fca75dcbb61571b, 99ba8d46c261419aa0813e9e10dc6cd1; no voters:
I20260807 18:33:15.050472 13878 raft_consensus.cc:2781] T 9415b291276b458380f53455ce4cd92e P 99ba8d46c261419aa0813e9e10dc6cd1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:33:15.050555 13878 raft_consensus.cc:482] T 9415b291276b458380f53455ce4cd92e P 99ba8d46c261419aa0813e9e10dc6cd1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:33:15.050593 13878 raft_consensus.cc:3037] T 9415b291276b458380f53455ce4cd92e P 99ba8d46c261419aa0813e9e10dc6cd1 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:33:15.051661 13878 raft_consensus.cc:504] T 9415b291276b458380f53455ce4cd92e P 99ba8d46c261419aa0813e9e10dc6cd1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97e3153b9b104e5a9fca75dcbb61571b" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 45267 } } peers { permanent_uuid: "99ba8d46c261419aa0813e9e10dc6cd1" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 45171 } } peers { permanent_uuid: "3596f9392f8746a09722d927914e3f20" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 41295 } }
I20260807 18:33:15.052026 13878 leader_election.cc:288] T 9415b291276b458380f53455ce4cd92e P 99ba8d46c261419aa0813e9e10dc6cd1 [CANDIDATE]: Term 1 election: Requested vote from peers 97e3153b9b104e5a9fca75dcbb61571b (127.11.14.65:45267), 3596f9392f8746a09722d927914e3f20 (127.11.14.67:41295)
I20260807 18:33:15.052386 13391 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9415b291276b458380f53455ce4cd92e" candidate_uuid: "99ba8d46c261419aa0813e9e10dc6cd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97e3153b9b104e5a9fca75dcbb61571b"
I20260807 18:33:15.052515 13391 raft_consensus.cc:3037] T 9415b291276b458380f53455ce4cd92e P 97e3153b9b104e5a9fca75dcbb61571b [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:33:15.052501 13695 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9415b291276b458380f53455ce4cd92e" candidate_uuid: "99ba8d46c261419aa0813e9e10dc6cd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3596f9392f8746a09722d927914e3f20"
I20260807 18:33:15.052649 13695 raft_consensus.cc:3037] T 9415b291276b458380f53455ce4cd92e P 3596f9392f8746a09722d927914e3f20 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:33:15.053483 13391 raft_consensus.cc:2454] T 9415b291276b458380f53455ce4cd92e P 97e3153b9b104e5a9fca75dcbb61571b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99ba8d46c261419aa0813e9e10dc6cd1 in term 1.
I20260807 18:33:15.053707 13695 raft_consensus.cc:2454] T 9415b291276b458380f53455ce4cd92e P 3596f9392f8746a09722d927914e3f20 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99ba8d46c261419aa0813e9e10dc6cd1 in term 1.
I20260807 18:33:15.053802 13470 leader_election.cc:302] T 9415b291276b458380f53455ce4cd92e P 99ba8d46c261419aa0813e9e10dc6cd1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 97e3153b9b104e5a9fca75dcbb61571b, 99ba8d46c261419aa0813e9e10dc6cd1; no voters:
I20260807 18:33:15.054013 13878 raft_consensus.cc:2781] T 9415b291276b458380f53455ce4cd92e P 99ba8d46c261419aa0813e9e10dc6cd1 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:33:15.054247 13878 raft_consensus.cc:686] T 9415b291276b458380f53455ce4cd92e P 99ba8d46c261419aa0813e9e10dc6cd1 [term 1 LEADER]: Becoming Leader. State: Replica: 99ba8d46c261419aa0813e9e10dc6cd1, State: Running, Role: LEADER
I20260807 18:33:15.054553 13878 consensus_queue.cc:237] T 9415b291276b458380f53455ce4cd92e P 99ba8d46c261419aa0813e9e10dc6cd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97e3153b9b104e5a9fca75dcbb61571b" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 45267 } } peers { permanent_uuid: "99ba8d46c261419aa0813e9e10dc6cd1" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 45171 } } peers { permanent_uuid: "3596f9392f8746a09722d927914e3f20" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 41295 } }
I20260807 18:33:15.056978 13218 catalog_manager.cc:5495] T 9415b291276b458380f53455ce4cd92e P 99ba8d46c261419aa0813e9e10dc6cd1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 99ba8d46c261419aa0813e9e10dc6cd1 (127.11.14.66). New cstate: current_term: 1 leader_uuid: "99ba8d46c261419aa0813e9e10dc6cd1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97e3153b9b104e5a9fca75dcbb61571b" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 45267 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "99ba8d46c261419aa0813e9e10dc6cd1" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 45171 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3596f9392f8746a09722d927914e3f20" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 41295 } health_report { overall_health: UNKNOWN } } }
I20260807 18:33:15.085577 13886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:15.081613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55462 (local address 127.11.14.66:45171)
0807 18:33:15.081920 (+ 307us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:15.081927 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:15.081941 (+ 14us) server_negotiation.cc:407] Connection header received
0807 18:33:15.082036 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:15.082038 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:15.082101 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:15.082218 (+ 117us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:33:15.082470 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:15.083126 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:15.084058 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:15.084298 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:15.084387 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:15.084634 (+ 247us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:33:15.085149 (+ 515us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:33:15.085242 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:15.085306 (+ 64us) server_negotiation.cc:299] Negotiation successful
0807 18:33:15.085360 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":229,"thread_start_us":139,"threads_started":1}
I20260807 18:33:15.097007 13889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:15.093257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.67:41295 (local address 127.11.14.66:47255)
0807 18:33:15.093597 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:15.093612 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:15.093715 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:15.093962 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:15.093965 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:15.093972 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 18:33:15.094089 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:15.094094 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:15.094849 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:15.094852 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:15.095634 (+ 782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:15.095645 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:15.096143 (+ 498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:15.096146 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:15.096285 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:15.096338 (+ 53us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 18:33:15.096713 (+ 375us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 18:33:15.096720 (+ 7us) client_negotiation.cc:769] Sending connection context
0807 18:33:15.096781 (+ 61us) client_negotiation.cc:240] Negotiation successful
0807 18:33:15.096858 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":129,"threads_started":1}
I20260807 18:33:15.097110 13882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:15.093479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.66:47255 (local address 127.11.14.67:41295)
0807 18:33:15.093601 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:15.093604 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:15.093678 (+ 74us) server_negotiation.cc:407] Connection header received
0807 18:33:15.093809 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:15.093811 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:15.093859 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:15.093924 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:33:15.094204 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:15.094703 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:15.095773 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:15.096020 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:15.096103 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:15.096503 (+ 400us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:33:15.096571 (+ 68us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:33:15.096641 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:15.096899 (+ 258us) server_negotiation.cc:299] Negotiation successful
0807 18:33:15.096955 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260807 18:33:15.103655 13695 raft_consensus.cc:1261] T 9415b291276b458380f53455ce4cd92e P 3596f9392f8746a09722d927914e3f20 [term 1 FOLLOWER]: Refusing update from remote peer 99ba8d46c261419aa0813e9e10dc6cd1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:33:15.103645 13391 raft_consensus.cc:1261] T 9415b291276b458380f53455ce4cd92e P 97e3153b9b104e5a9fca75dcbb61571b [term 1 FOLLOWER]: Refusing update from remote peer 99ba8d46c261419aa0813e9e10dc6cd1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:33:15.104195 13878 consensus_queue.cc:1035] T 9415b291276b458380f53455ce4cd92e P 99ba8d46c261419aa0813e9e10dc6cd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3596f9392f8746a09722d927914e3f20" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 41295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:33:15.104414 13883 consensus_queue.cc:1035] T 9415b291276b458380f53455ce4cd92e P 99ba8d46c261419aa0813e9e10dc6cd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "97e3153b9b104e5a9fca75dcbb61571b" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 45267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:33:15.123986 13886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:15.120321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.67:42537 (local address 127.11.14.66:45171)
0807 18:33:15.120452 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:15.120457 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:15.120812 (+ 355us) server_negotiation.cc:407] Connection header received
0807 18:33:15.120866 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:15.120868 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:15.120910 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:15.120976 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:33:15.121244 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:15.121830 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:15.122812 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:15.123134 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:15.123246 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:15.123534 (+ 288us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:33:15.123668 (+ 134us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:33:15.123728 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:15.123823 (+ 95us) server_negotiation.cc:299] Negotiation successful
0807 18:33:15.123877 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260807 18:33:15.124003 13894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:15.120227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.66:45171 (local address 127.11.14.67:42537)
0807 18:33:15.120601 (+ 374us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:15.120614 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:15.120753 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:15.120996 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:15.120998 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:15.121005 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 18:33:15.121141 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:15.121147 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:15.121961 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:15.121964 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:15.122677 (+ 713us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:15.122688 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:15.123256 (+ 568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:15.123261 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:15.123365 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:15.123400 (+ 35us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 18:33:15.123749 (+ 349us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 18:33:15.123755 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 18:33:15.123815 (+ 60us) client_negotiation.cc:240] Negotiation successful
0807 18:33:15.123887 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":122,"threads_started":1}
I20260807 18:33:15.147842 13220 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54842:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:33:15.148027 13220 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54842:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:33:15.150161 13220 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e09adbeddf294788bec26d0474bcf37c: Sending DeleteTablet for 3 replicas of tablet 9415b291276b458380f53455ce4cd92e
I20260807 18:33:15.150729 13371 tablet_service.cc:1514] Processing DeleteTablet for tablet 9415b291276b458380f53455ce4cd92e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:33:15 UTC) from {username='kudu', principal='kudu/127.11.14.126@KRBTEST.COM'} at 127.0.0.1:36712
I20260807 18:33:15.150853 13515 tablet_service.cc:1514] Processing DeleteTablet for tablet 9415b291276b458380f53455ce4cd92e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:33:15 UTC) from {username='kudu', principal='kudu/127.11.14.126@KRBTEST.COM'} at 127.0.0.1:55458
I20260807 18:33:15.150954 13862 tablet_replica.cc:330] T 9415b291276b458380f53455ce4cd92e P 97e3153b9b104e5a9fca75dcbb61571b: stopping tablet replica
I20260807 18:33:15.151005 13860 tablet_replica.cc:330] T 9415b291276b458380f53455ce4cd92e P 99ba8d46c261419aa0813e9e10dc6cd1: stopping tablet replica
I20260807 18:33:15.150957 13672 tablet_service.cc:1514] Processing DeleteTablet for tablet 9415b291276b458380f53455ce4cd92e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:33:15 UTC) from {username='kudu', principal='kudu/127.11.14.126@KRBTEST.COM'} at 127.0.0.1:50110
I20260807 18:33:15.151069 13862 raft_consensus.cc:2229] T 9415b291276b458380f53455ce4cd92e P 97e3153b9b104e5a9fca75dcbb61571b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:33:15.151099 13860 raft_consensus.cc:2229] T 9415b291276b458380f53455ce4cd92e P 99ba8d46c261419aa0813e9e10dc6cd1 [term 1 LEADER]: Raft consensus shutting down.
I20260807 18:33:15.151144 13863 tablet_replica.cc:330] T 9415b291276b458380f53455ce4cd92e P 3596f9392f8746a09722d927914e3f20: stopping tablet replica
I20260807 18:33:15.151187 13862 raft_consensus.cc:2258] T 9415b291276b458380f53455ce4cd92e P 97e3153b9b104e5a9fca75dcbb61571b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:33:15.151281 13863 raft_consensus.cc:2229] T 9415b291276b458380f53455ce4cd92e P 3596f9392f8746a09722d927914e3f20 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:33:15.151392 13863 raft_consensus.cc:2258] T 9415b291276b458380f53455ce4cd92e P 3596f9392f8746a09722d927914e3f20 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:33:15.151381 13860 raft_consensus.cc:2258] T 9415b291276b458380f53455ce4cd92e P 99ba8d46c261419aa0813e9e10dc6cd1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:33:15.152501 13862 ts_tablet_manager.cc:1905] T 9415b291276b458380f53455ce4cd92e P 97e3153b9b104e5a9fca75dcbb61571b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:33:15.152518 13220 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54842:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 18:33:15.152921 13220 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 18:33:15.155268 13862 ts_tablet_manager.cc:1918] T 9415b291276b458380f53455ce4cd92e P 97e3153b9b104e5a9fca75dcbb61571b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 18:33:15.155362 13862 log.cc:1196] T 9415b291276b458380f53455ce4cd92e P 97e3153b9b104e5a9fca75dcbb61571b: Deleting WAL directory at /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127579900523-11321-0/minicluster-data/ts-0/wal/wals/9415b291276b458380f53455ce4cd92e
I20260807 18:33:15.155683 13862 ts_tablet_manager.cc:1939] T 9415b291276b458380f53455ce4cd92e P 97e3153b9b104e5a9fca75dcbb61571b: Deleting consensus metadata
I20260807 18:33:15.156294 13863 ts_tablet_manager.cc:1905] T 9415b291276b458380f53455ce4cd92e P 3596f9392f8746a09722d927914e3f20: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:33:15.158209 13672 tablet_service.cc:1467] Processing CreateTablet for tablet 94402f582bcf4e65b1296d97910d9356 (DEFAULT_TABLE table=test-table [id=680c862e686b478db3e9f5701de90464]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:33:15.158481 13672 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 94402f582bcf4e65b1296d97910d9356. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:33:15.158490 13515 tablet_service.cc:1467] Processing CreateTablet for tablet 94402f582bcf4e65b1296d97910d9356 (DEFAULT_TABLE table=test-table [id=680c862e686b478db3e9f5701de90464]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:33:15.158776 13515 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 94402f582bcf4e65b1296d97910d9356. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:33:15.158943 13371 tablet_service.cc:1467] Processing CreateTablet for tablet 94402f582bcf4e65b1296d97910d9356 (DEFAULT_TABLE table=test-table [id=680c862e686b478db3e9f5701de90464]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:33:15.159174 13371 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 94402f582bcf4e65b1296d97910d9356. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:33:15.160776 13874 tablet_bootstrap.cc:492] T 94402f582bcf4e65b1296d97910d9356 P 3596f9392f8746a09722d927914e3f20: Bootstrap starting.
I20260807 18:33:15.161192 13872 tablet_bootstrap.cc:492] T 94402f582bcf4e65b1296d97910d9356 P 99ba8d46c261419aa0813e9e10dc6cd1: Bootstrap starting.
I20260807 18:33:15.161603 13874 tablet_bootstrap.cc:654] T 94402f582bcf4e65b1296d97910d9356 P 3596f9392f8746a09722d927914e3f20: Neither blocks nor log segments found. Creating new log.
I20260807 18:33:15.162112 13872 tablet_bootstrap.cc:654] T 94402f582bcf4e65b1296d97910d9356 P 99ba8d46c261419aa0813e9e10dc6cd1: Neither blocks nor log segments found. Creating new log.
I20260807 18:33:15.162557 13873 tablet_bootstrap.cc:492] T 94402f582bcf4e65b1296d97910d9356 P 97e3153b9b104e5a9fca75dcbb61571b: Bootstrap starting.
I20260807 18:33:15.163033 13874 tablet_bootstrap.cc:492] T 94402f582bcf4e65b1296d97910d9356 P 3596f9392f8746a09722d927914e3f20: No bootstrap required, opened a new log
I20260807 18:33:15.163125 13874 ts_tablet_manager.cc:1397] T 94402f582bcf4e65b1296d97910d9356 P 3596f9392f8746a09722d927914e3f20: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 18:33:15.163429 13873 tablet_bootstrap.cc:654] T 94402f582bcf4e65b1296d97910d9356 P 97e3153b9b104e5a9fca75dcbb61571b: Neither blocks nor log segments found. Creating new log.
I20260807 18:33:15.163590 13874 raft_consensus.cc:348] T 94402f582bcf4e65b1296d97910d9356 P 3596f9392f8746a09722d927914e3f20 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97e3153b9b104e5a9fca75dcbb61571b" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 45267 } } peers { permanent_uuid: "99ba8d46c261419aa0813e9e10dc6cd1" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 45171 } } peers { permanent_uuid: "3596f9392f8746a09722d927914e3f20" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 41295 } }
I20260807 18:33:15.163713 13874 raft_consensus.cc:374] T 94402f582bcf4e65b1296d97910d9356 P 3596f9392f8746a09722d927914e3f20 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:33:15.163762 13874 raft_consensus.cc:729] T 94402f582bcf4e65b1296d97910d9356 P 3596f9392f8746a09722d927914e3f20 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3596f9392f8746a09722d927914e3f20, State: Initialized, Role: FOLLOWER
I20260807 18:33:15.163903 13874 consensus_queue.cc:260] T 94402f582bcf4e65b1296d97910d9356 P 3596f9392f8746a09722d927914e3f20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97e3153b9b104e5a9fca75dcbb61571b" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 45267 } } peers { permanent_uuid: "99ba8d46c261419aa0813e9e10dc6cd1" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 45171 } } peers { permanent_uuid: "3596f9392f8746a09722d927914e3f20" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 41295 } }
I20260807 18:33:15.164213 13874 ts_tablet_manager.cc:1428] T 94402f582bcf4e65b1296d97910d9356 P 3596f9392f8746a09722d927914e3f20: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 18:33:15.165066 13860 ts_tablet_manager.cc:1905] T 9415b291276b458380f53455ce4cd92e P 99ba8d46c261419aa0813e9e10dc6cd1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:33:15.165163 13873 tablet_bootstrap.cc:492] T 94402f582bcf4e65b1296d97910d9356 P 97e3153b9b104e5a9fca75dcbb61571b: No bootstrap required, opened a new log
I20260807 18:33:15.165258 13873 ts_tablet_manager.cc:1397] T 94402f582bcf4e65b1296d97910d9356 P 97e3153b9b104e5a9fca75dcbb61571b: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260807 18:33:15.165681 13873 raft_consensus.cc:348] T 94402f582bcf4e65b1296d97910d9356 P 97e3153b9b104e5a9fca75dcbb61571b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97e3153b9b104e5a9fca75dcbb61571b" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 45267 } } peers { permanent_uuid: "99ba8d46c261419aa0813e9e10dc6cd1" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 45171 } } peers { permanent_uuid: "3596f9392f8746a09722d927914e3f20" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 41295 } }
I20260807 18:33:15.165792 13873 raft_consensus.cc:374] T 94402f582bcf4e65b1296d97910d9356 P 97e3153b9b104e5a9fca75dcbb61571b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:33:15.165827 13873 raft_consensus.cc:729] T 94402f582bcf4e65b1296d97910d9356 P 97e3153b9b104e5a9fca75dcbb61571b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97e3153b9b104e5a9fca75dcbb61571b, State: Initialized, Role: FOLLOWER
I20260807 18:33:15.165951 13873 consensus_queue.cc:260] T 94402f582bcf4e65b1296d97910d9356 P 97e3153b9b104e5a9fca75dcbb61571b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97e3153b9b104e5a9fca75dcbb61571b" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 45267 } } peers { permanent_uuid: "99ba8d46c261419aa0813e9e10dc6cd1" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 45171 } } peers { permanent_uuid: "3596f9392f8746a09722d927914e3f20" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 41295 } }
I20260807 18:33:15.166218 13873 ts_tablet_manager.cc:1428] T 94402f582bcf4e65b1296d97910d9356 P 97e3153b9b104e5a9fca75dcbb61571b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 18:33:15.167253 13863 ts_tablet_manager.cc:1918] T 9415b291276b458380f53455ce4cd92e P 3596f9392f8746a09722d927914e3f20: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 18:33:15.167338 13863 log.cc:1196] T 9415b291276b458380f53455ce4cd92e P 3596f9392f8746a09722d927914e3f20: Deleting WAL directory at /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127579900523-11321-0/minicluster-data/ts-2/wal/wals/9415b291276b458380f53455ce4cd92e
I20260807 18:33:15.167863 13863 ts_tablet_manager.cc:1939] T 9415b291276b458380f53455ce4cd92e P 3596f9392f8746a09722d927914e3f20: Deleting consensus metadata
I20260807 18:33:15.168295 13860 ts_tablet_manager.cc:1918] T 9415b291276b458380f53455ce4cd92e P 99ba8d46c261419aa0813e9e10dc6cd1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 18:33:15.168380 13860 log.cc:1196] T 9415b291276b458380f53455ce4cd92e P 99ba8d46c261419aa0813e9e10dc6cd1: Deleting WAL directory at /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127579900523-11321-0/minicluster-data/ts-1/wal/wals/9415b291276b458380f53455ce4cd92e
I20260807 18:33:15.168478 13205 catalog_manager.cc:4841] TS 3596f9392f8746a09722d927914e3f20 (127.11.14.67:41295): tablet 9415b291276b458380f53455ce4cd92e (table test-table [id=44af5ca7d92447a6a55ae7ea2d5125e2]) successfully deleted
I20260807 18:33:15.168684 13860 ts_tablet_manager.cc:1939] T 9415b291276b458380f53455ce4cd92e P 99ba8d46c261419aa0813e9e10dc6cd1: Deleting consensus metadata
I20260807 18:33:15.169041 13206 catalog_manager.cc:4841] TS 97e3153b9b104e5a9fca75dcbb61571b (127.11.14.65:45267): tablet 9415b291276b458380f53455ce4cd92e (table test-table [id=44af5ca7d92447a6a55ae7ea2d5125e2]) successfully deleted
I20260807 18:33:15.169378 13207 catalog_manager.cc:4841] TS 99ba8d46c261419aa0813e9e10dc6cd1 (127.11.14.66:45171): tablet 9415b291276b458380f53455ce4cd92e (table test-table [id=44af5ca7d92447a6a55ae7ea2d5125e2]) successfully deleted
I20260807 18:33:15.169684 13872 tablet_bootstrap.cc:492] T 94402f582bcf4e65b1296d97910d9356 P 99ba8d46c261419aa0813e9e10dc6cd1: No bootstrap required, opened a new log
I20260807 18:33:15.169770 13872 ts_tablet_manager.cc:1397] T 94402f582bcf4e65b1296d97910d9356 P 99ba8d46c261419aa0813e9e10dc6cd1: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.000s
I20260807 18:33:15.170164 13872 raft_consensus.cc:348] T 94402f582bcf4e65b1296d97910d9356 P 99ba8d46c261419aa0813e9e10dc6cd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97e3153b9b104e5a9fca75dcbb61571b" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 45267 } } peers { permanent_uuid: "99ba8d46c261419aa0813e9e10dc6cd1" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 45171 } } peers { permanent_uuid: "3596f9392f8746a09722d927914e3f20" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 41295 } }
I20260807 18:33:15.170290 13872 raft_consensus.cc:374] T 94402f582bcf4e65b1296d97910d9356 P 99ba8d46c261419aa0813e9e10dc6cd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:33:15.170338 13872 raft_consensus.cc:729] T 94402f582bcf4e65b1296d97910d9356 P 99ba8d46c261419aa0813e9e10dc6cd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99ba8d46c261419aa0813e9e10dc6cd1, State: Initialized, Role: FOLLOWER
I20260807 18:33:15.170475 13872 consensus_queue.cc:260] T 94402f582bcf4e65b1296d97910d9356 P 99ba8d46c261419aa0813e9e10dc6cd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97e3153b9b104e5a9fca75dcbb61571b" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 45267 } } peers { permanent_uuid: "99ba8d46c261419aa0813e9e10dc6cd1" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 45171 } } peers { permanent_uuid: "3596f9392f8746a09722d927914e3f20" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 41295 } }
I20260807 18:33:15.170749 13872 ts_tablet_manager.cc:1428] T 94402f582bcf4e65b1296d97910d9356 P 99ba8d46c261419aa0813e9e10dc6cd1: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260807 18:33:15.294575 13851 raft_consensus.cc:482] T 94402f582bcf4e65b1296d97910d9356 P 97e3153b9b104e5a9fca75dcbb61571b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:33:15.294771 13851 raft_consensus.cc:504] T 94402f582bcf4e65b1296d97910d9356 P 97e3153b9b104e5a9fca75dcbb61571b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97e3153b9b104e5a9fca75dcbb61571b" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 45267 } } peers { permanent_uuid: "99ba8d46c261419aa0813e9e10dc6cd1" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 45171 } } peers { permanent_uuid: "3596f9392f8746a09722d927914e3f20" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 41295 } }
I20260807 18:33:15.295248 13851 leader_election.cc:288] T 94402f582bcf4e65b1296d97910d9356 P 97e3153b9b104e5a9fca75dcbb61571b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99ba8d46c261419aa0813e9e10dc6cd1 (127.11.14.66:45171), 3596f9392f8746a09722d927914e3f20 (127.11.14.67:41295)
I20260807 18:33:15.295864 13541 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "94402f582bcf4e65b1296d97910d9356" candidate_uuid: "97e3153b9b104e5a9fca75dcbb61571b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99ba8d46c261419aa0813e9e10dc6cd1" is_pre_election: true
I20260807 18:33:15.295857 13695 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "94402f582bcf4e65b1296d97910d9356" candidate_uuid: "97e3153b9b104e5a9fca75dcbb61571b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3596f9392f8746a09722d927914e3f20" is_pre_election: true
I20260807 18:33:15.296034 13695 raft_consensus.cc:2454] T 94402f582bcf4e65b1296d97910d9356 P 3596f9392f8746a09722d927914e3f20 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 97e3153b9b104e5a9fca75dcbb61571b in term 0.
I20260807 18:33:15.296042 13541 raft_consensus.cc:2454] T 94402f582bcf4e65b1296d97910d9356 P 99ba8d46c261419aa0813e9e10dc6cd1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 97e3153b9b104e5a9fca75dcbb61571b in term 0.
I20260807 18:33:15.296357 13317 leader_election.cc:302] T 94402f582bcf4e65b1296d97910d9356 P 97e3153b9b104e5a9fca75dcbb61571b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3596f9392f8746a09722d927914e3f20, 97e3153b9b104e5a9fca75dcbb61571b; no voters:
I20260807 18:33:15.296741 13851 raft_consensus.cc:2781] T 94402f582bcf4e65b1296d97910d9356 P 97e3153b9b104e5a9fca75dcbb61571b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:33:15.296821 13851 raft_consensus.cc:482] T 94402f582bcf4e65b1296d97910d9356 P 97e3153b9b104e5a9fca75dcbb61571b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:33:15.296875 13851 raft_consensus.cc:3037] T 94402f582bcf4e65b1296d97910d9356 P 97e3153b9b104e5a9fca75dcbb61571b [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:33:15.297972 13851 raft_consensus.cc:504] T 94402f582bcf4e65b1296d97910d9356 P 97e3153b9b104e5a9fca75dcbb61571b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97e3153b9b104e5a9fca75dcbb61571b" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 45267 } } peers { permanent_uuid: "99ba8d46c261419aa0813e9e10dc6cd1" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 45171 } } peers { permanent_uuid: "3596f9392f8746a09722d927914e3f20" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 41295 } }
I20260807 18:33:15.298377 13851 leader_election.cc:288] T 94402f582bcf4e65b1296d97910d9356 P 97e3153b9b104e5a9fca75dcbb61571b [CANDIDATE]: Term 1 election: Requested vote from peers 99ba8d46c261419aa0813e9e10dc6cd1 (127.11.14.66:45171), 3596f9392f8746a09722d927914e3f20 (127.11.14.67:41295)
I20260807 18:33:15.298755 13695 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "94402f582bcf4e65b1296d97910d9356" candidate_uuid: "97e3153b9b104e5a9fca75dcbb61571b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3596f9392f8746a09722d927914e3f20"
I20260807 18:33:15.298782 13541 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "94402f582bcf4e65b1296d97910d9356" candidate_uuid: "97e3153b9b104e5a9fca75dcbb61571b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99ba8d46c261419aa0813e9e10dc6cd1"
I20260807 18:33:15.298909 13695 raft_consensus.cc:3037] T 94402f582bcf4e65b1296d97910d9356 P 3596f9392f8746a09722d927914e3f20 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:33:15.298920 13541 raft_consensus.cc:3037] T 94402f582bcf4e65b1296d97910d9356 P 99ba8d46c261419aa0813e9e10dc6cd1 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:33:15.300045 13695 raft_consensus.cc:2454] T 94402f582bcf4e65b1296d97910d9356 P 3596f9392f8746a09722d927914e3f20 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 97e3153b9b104e5a9fca75dcbb61571b in term 1.
I20260807 18:33:15.300154 13541 raft_consensus.cc:2454] T 94402f582bcf4e65b1296d97910d9356 P 99ba8d46c261419aa0813e9e10dc6cd1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 97e3153b9b104e5a9fca75dcbb61571b in term 1.
I20260807 18:33:15.300359 13317 leader_election.cc:302] T 94402f582bcf4e65b1296d97910d9356 P 97e3153b9b104e5a9fca75dcbb61571b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3596f9392f8746a09722d927914e3f20, 97e3153b9b104e5a9fca75dcbb61571b; no voters:
I20260807 18:33:15.300534 13851 raft_consensus.cc:2781] T 94402f582bcf4e65b1296d97910d9356 P 97e3153b9b104e5a9fca75dcbb61571b [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:33:15.300622 13851 raft_consensus.cc:686] T 94402f582bcf4e65b1296d97910d9356 P 97e3153b9b104e5a9fca75dcbb61571b [term 1 LEADER]: Becoming Leader. State: Replica: 97e3153b9b104e5a9fca75dcbb61571b, State: Running, Role: LEADER
I20260807 18:33:15.300786 13851 consensus_queue.cc:237] T 94402f582bcf4e65b1296d97910d9356 P 97e3153b9b104e5a9fca75dcbb61571b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97e3153b9b104e5a9fca75dcbb61571b" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 45267 } } peers { permanent_uuid: "99ba8d46c261419aa0813e9e10dc6cd1" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 45171 } } peers { permanent_uuid: "3596f9392f8746a09722d927914e3f20" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 41295 } }
I20260807 18:33:15.302718 13220 catalog_manager.cc:5495] T 94402f582bcf4e65b1296d97910d9356 P 97e3153b9b104e5a9fca75dcbb61571b reported cstate change: term changed from 0 to 1, leader changed from <none> to 97e3153b9b104e5a9fca75dcbb61571b (127.11.14.65). New cstate: current_term: 1 leader_uuid: "97e3153b9b104e5a9fca75dcbb61571b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97e3153b9b104e5a9fca75dcbb61571b" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 45267 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "99ba8d46c261419aa0813e9e10dc6cd1" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 45171 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3596f9392f8746a09722d927914e3f20" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 41295 } health_report { overall_health: UNKNOWN } } }
I20260807 18:33:15.331278 13881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:15.328498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36734 (local address 127.11.14.65:45267)
0807 18:33:15.328627 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:15.328630 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:15.328644 (+ 14us) server_negotiation.cc:407] Connection header received
0807 18:33:15.328770 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:15.328772 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:15.328817 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:15.328880 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:33:15.329094 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:15.329611 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:15.330421 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:15.330628 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:15.330705 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:15.330926 (+ 221us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:33:15.330990 (+ 64us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:33:15.331041 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:15.331118 (+ 77us) server_negotiation.cc:299] Negotiation successful
0807 18:33:15.331162 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260807 18:33:15.337751 13695 raft_consensus.cc:1261] T 94402f582bcf4e65b1296d97910d9356 P 3596f9392f8746a09722d927914e3f20 [term 1 FOLLOWER]: Refusing update from remote peer 97e3153b9b104e5a9fca75dcbb61571b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:33:15.337781 13541 raft_consensus.cc:1261] T 94402f582bcf4e65b1296d97910d9356 P 99ba8d46c261419aa0813e9e10dc6cd1 [term 1 FOLLOWER]: Refusing update from remote peer 97e3153b9b104e5a9fca75dcbb61571b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:33:15.338192 13838 consensus_queue.cc:1035] T 94402f582bcf4e65b1296d97910d9356 P 97e3153b9b104e5a9fca75dcbb61571b [LEADER]: Connected to new peer: Peer: permanent_uuid: "99ba8d46c261419aa0813e9e10dc6cd1" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 45171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:33:15.338445 13851 consensus_queue.cc:1035] T 94402f582bcf4e65b1296d97910d9356 P 97e3153b9b104e5a9fca75dcbb61571b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3596f9392f8746a09722d927914e3f20" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 41295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:33:15.341892 13843 mvcc.cc:205] Tried to move back new op lower bound from 7315978630500159488 to 7315978630353764352. Current Snapshot: MvccSnapshot[applied={T|T < 7315978630500159488}]
W20260807 18:33:15.352628 13843 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260807 18:33:16.903164 13219 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54842:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:33:16.903337 13219 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54842:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:33:16.905762 13219 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e09adbeddf294788bec26d0474bcf37c: Sending DeleteTablet for 3 replicas of tablet 94402f582bcf4e65b1296d97910d9356
I20260807 18:33:16.906450 13371 tablet_service.cc:1514] Processing DeleteTablet for tablet 94402f582bcf4e65b1296d97910d9356 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:33:16 UTC) from {username='kudu', principal='kudu/127.11.14.126@KRBTEST.COM'} at 127.0.0.1:36712
I20260807 18:33:16.906491 13515 tablet_service.cc:1514] Processing DeleteTablet for tablet 94402f582bcf4e65b1296d97910d9356 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:33:16 UTC) from {username='kudu', principal='kudu/127.11.14.126@KRBTEST.COM'} at 127.0.0.1:55458
I20260807 18:33:16.906505 13673 tablet_service.cc:1514] Processing DeleteTablet for tablet 94402f582bcf4e65b1296d97910d9356 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:33:16 UTC) from {username='kudu', principal='kudu/127.11.14.126@KRBTEST.COM'} at 127.0.0.1:50110
I20260807 18:33:16.906863 13930 tablet_replica.cc:330] T 94402f582bcf4e65b1296d97910d9356 P 3596f9392f8746a09722d927914e3f20: stopping tablet replica
I20260807 18:33:16.906869 13929 tablet_replica.cc:330] T 94402f582bcf4e65b1296d97910d9356 P 97e3153b9b104e5a9fca75dcbb61571b: stopping tablet replica
I20260807 18:33:16.906899 11321 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkz56XN/build/debug/bin/kudu with pid 13300
I20260807 18:33:16.910013 13930 raft_consensus.cc:2229] T 94402f582bcf4e65b1296d97910d9356 P 3596f9392f8746a09722d927914e3f20 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:33:16.910073 13928 tablet_replica.cc:330] T 94402f582bcf4e65b1296d97910d9356 P 99ba8d46c261419aa0813e9e10dc6cd1: stopping tablet replica
I20260807 18:33:16.910182 13930 raft_consensus.cc:2258] T 94402f582bcf4e65b1296d97910d9356 P 3596f9392f8746a09722d927914e3f20 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:33:16.910239 13928 raft_consensus.cc:2229] T 94402f582bcf4e65b1296d97910d9356 P 99ba8d46c261419aa0813e9e10dc6cd1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:33:16.910386 13928 raft_consensus.cc:2258] T 94402f582bcf4e65b1296d97910d9356 P 99ba8d46c261419aa0813e9e10dc6cd1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:33:16.910799 13930 ts_tablet_manager.cc:1905] T 94402f582bcf4e65b1296d97910d9356 P 3596f9392f8746a09722d927914e3f20: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:33:16.913116 13930 ts_tablet_manager.cc:1918] T 94402f582bcf4e65b1296d97910d9356 P 3596f9392f8746a09722d927914e3f20: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 18:33:16.913221 13930 log.cc:1196] T 94402f582bcf4e65b1296d97910d9356 P 3596f9392f8746a09722d927914e3f20: Deleting WAL directory at /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127579900523-11321-0/minicluster-data/ts-2/wal/wals/94402f582bcf4e65b1296d97910d9356
I20260807 18:33:16.913489 13930 ts_tablet_manager.cc:1939] T 94402f582bcf4e65b1296d97910d9356 P 3596f9392f8746a09722d927914e3f20: Deleting consensus metadata
I20260807 18:33:16.914259 13205 catalog_manager.cc:4841] TS 3596f9392f8746a09722d927914e3f20 (127.11.14.67:41295): tablet 94402f582bcf4e65b1296d97910d9356 (table test-table [id=680c862e686b478db3e9f5701de90464]) successfully deleted
I20260807 18:33:16.914990 13928 ts_tablet_manager.cc:1905] T 94402f582bcf4e65b1296d97910d9356 P 99ba8d46c261419aa0813e9e10dc6cd1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:33:16.919132 13928 ts_tablet_manager.cc:1918] T 94402f582bcf4e65b1296d97910d9356 P 99ba8d46c261419aa0813e9e10dc6cd1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 18:33:16.919298 13928 log.cc:1196] T 94402f582bcf4e65b1296d97910d9356 P 99ba8d46c261419aa0813e9e10dc6cd1: Deleting WAL directory at /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786127579900523-11321-0/minicluster-data/ts-1/wal/wals/94402f582bcf4e65b1296d97910d9356
W20260807 18:33:16.919423 13206 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.14.65:45267 (error 108)
I20260807 18:33:16.919631 11321 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkz56XN/build/debug/bin/kudu with pid 13453
I20260807 18:33:16.919706 13928 ts_tablet_manager.cc:1939] T 94402f582bcf4e65b1296d97910d9356 P 99ba8d46c261419aa0813e9e10dc6cd1: Deleting consensus metadata
I20260807 18:33:16.923305 13932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:16.922706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.65:45267 (local address 127.0.0.1:36740)
0807 18:33:16.923041 (+ 335us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:16.923147 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.65:45267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":227,"thread_start_us":92,"threads_started":1}
W20260807 18:33:16.923509 13206 catalog_manager.cc:4584] TS 97e3153b9b104e5a9fca75dcbb61571b (127.11.14.65:45267): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 94402f582bcf4e65b1296d97910d9356: Network error: Client connection negotiation failed: client connection to 127.11.14.65:45267: connect: Connection refused (error 111)
I20260807 18:33:16.931643 11321 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkz56XN/build/debug/bin/kudu with pid 13600
I20260807 18:33:16.934302 13932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:16.931765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.66:45171 (local address 127.0.0.1:55470)
0807 18:33:16.934154 (+ 2389us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:16.934209 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.66:45171: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2292}
I20260807 18:33:16.942103 11321 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkz56XN/build/debug/bin/kudu with pid 13189
2026-08-07T18:33:16Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (6947 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127579900523-11321-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:33:17 dist-test-slave-6k22 krb5kdc[13935](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:33:17 dist-test-slave-6k22 krb5kdc[13935](info): set up 2 sockets
Aug 07 18:33:17 dist-test-slave-6k22 krb5kdc[13935](info): commencing operation
krb5kdc: starting...
W20260807 18:33:18.074765 11321 mini_kdc.cc:121] Time spent starting KDC: real 1.107s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127579900523-11321-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127579900523-11321-0/krb5kdc/test-user.keytab.
Aug 07 18:33:18 dist-test-slave-6k22 krb5kdc[13935](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127598, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:33:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:33:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.14.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.14.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.14.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.126.keytab.
Entry for principal kudu/127.11.14.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.126.keytab.
I20260807 18:33:18.214963 11321 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkz56XN/build/debug/bin/kudu
/tmp/dist-test-taskkz56XN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127579900523-11321-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127579900523-11321-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127579900523-11321-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127579900523-11321-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.14.126:44469
--webserver_interface=127.11.14.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.126.keytab
--principal=kudu/127.11.14.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.14.84:41087
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.14.126:44469
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127579900523-11321-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127579900523-11321-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127579900523-11321-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:33:18.338548 13950 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:33:18.338830 13950 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:33:18.338872 13950 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:33:18.343124 13950 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:33:18.343187 13950 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:33:18.343207 13950 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:33:18.343225 13950 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:33:18.343271 13950 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:33:18.348666 13950 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.14.84:41087
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127579900523-11321-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127579900523-11321-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.14.126:44469
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.126.keytab
--principal=kudu/127.11.14.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.14.126:44469
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127579900523-11321-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.14.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13950
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127579900523-11321-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:33:18.349802 13950 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:33:18.350878 13950 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 18:33:18.360268 13950 server_base.cc:1034] running on GCE node
W20260807 18:33:18.360258 13955 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:33:18.360448 13956 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:33:18.360258 13958 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:33:18.361126 13950 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:33:18.362401 13950 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:33:18.363615 13950 hybrid_clock.cc:648] HybridClock initialized: now 1786127598363571 us; error 46 us; skew 500 ppm
Aug 07 18:33:18 dist-test-slave-6k22 krb5kdc[13935](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127598, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:33:18.366559 13950 init.cc:377] Logged in from keytab as kudu/127.11.14.126@KRBTEST.COM (short username kudu)
I20260807 18:33:18.367619 13950 webserver.cc:460] Webserver started at http://127.11.14.126:35655/ using document root <none> and password file <none>
I20260807 18:33:18.368224 13950 fs_manager.cc:359] Metadata directory not provided
I20260807 18:33:18.368299 13950 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:33:18.368535 13950 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:33:18.370697 13950 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127579900523-11321-0/minicluster-data/master-0/data/instance:
uuid: "44aad0f27da7493788209ac04f103181"
format_stamp: "Formatted at 2026-08-07 18:33:18 on dist-test-slave-6k22"
server_key: "08ecb2e08097ae93078061c45059c6f4"
server_key_iv: "03f9a468eb3354c0c5e2001039ad7f9f"
server_key_version: "encryptionkey@0"
I20260807 18:33:18.371297 13950 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127579900523-11321-0/minicluster-data/master-0/wal/instance:
uuid: "44aad0f27da7493788209ac04f103181"
format_stamp: "Formatted at 2026-08-07 18:33:18 on dist-test-slave-6k22"
server_key: "08ecb2e08097ae93078061c45059c6f4"
server_key_iv: "03f9a468eb3354c0c5e2001039ad7f9f"
server_key_version: "encryptionkey@0"
I20260807 18:33:18.375293 13950 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260807 18:33:18.377943 13965 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:33:18.379390 13950 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 18:33:18.379580 13950 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127579900523-11321-0/minicluster-data/master-0/data,/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127579900523-11321-0/minicluster-data/master-0/wal
uuid: "44aad0f27da7493788209ac04f103181"
format_stamp: "Formatted at 2026-08-07 18:33:18 on dist-test-slave-6k22"
server_key: "08ecb2e08097ae93078061c45059c6f4"
server_key_iv: "03f9a468eb3354c0c5e2001039ad7f9f"
server_key_version: "encryptionkey@0"
I20260807 18:33:18.379776 13950 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127579900523-11321-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127579900523-11321-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127579900523-11321-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:33:18.398387 13950 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:33:18.402411 13950 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:33:18.402697 13950 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:33:18.411403 13950 rpc_server.cc:307] RPC server started. Bound to: 127.11.14.126:44469
I20260807 18:33:18.411413 14021 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.14.126:44469 every 8 connection(s)
I20260807 18:33:18.412536 13950 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127579900523-11321-0/minicluster-data/master-0/data/info.pb
I20260807 18:33:18.412981 11321 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkz56XN/build/debug/bin/kudu as pid 13950
I20260807 18:33:18.413108 11321 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127579900523-11321-0/minicluster-data/master-0/wal/instance
I20260807 18:33:18.413470 11321 external_mini_cluster.cc:1428] Setting key 22c698caaabd84b92daa4bee7a73ecde
I20260807 18:33:18.416903 14022 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 07 18:33:18 dist-test-slave-6k22 krb5kdc[13935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127598, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.14.126@KRBTEST.COM
I20260807 18:33:18.423347 14022 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 44aad0f27da7493788209ac04f103181: Bootstrap starting.
I20260807 18:33:18.426187 14022 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 44aad0f27da7493788209ac04f103181: Neither blocks nor log segments found. Creating new log.
I20260807 18:33:18.427239 14022 log.cc:824] T 00000000000000000000000000000000 P 44aad0f27da7493788209ac04f103181: Log is configured to *not* fsync() on all Append() calls
I20260807 18:33:18.429015 14025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:18.414934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35016 (local address 127.11.14.126:44469)
0807 18:33:18.415383 (+ 449us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:18.415400 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:18.415441 (+ 41us) server_negotiation.cc:407] Connection header received
0807 18:33:18.416207 (+ 766us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:18.416227 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:18.416590 (+ 363us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:18.416972 (+ 382us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:33:18.417892 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:18.419026 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:18.420059 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:18.420606 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:18.420806 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:18.423614 (+ 2808us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:33:18.423624 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:33:18.423645 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:33:18.423676 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:33:18.425607 (+ 1931us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:18.426125 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:18.426131 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:18.426137 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:18.426267 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:18.426684 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:18.426687 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:18.426688 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:18.427114 (+ 426us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:33:18.427289 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:18.427776 (+ 487us) server_negotiation.cc:299] Negotiation successful
0807 18:33:18.428151 (+ 375us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":299,"thread_start_us":134,"threads_started":1}
I20260807 18:33:18.430387 14022 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 44aad0f27da7493788209ac04f103181: No bootstrap required, opened a new log
I20260807 18:33:18.434089 14022 raft_consensus.cc:348] T 00000000000000000000000000000000 P 44aad0f27da7493788209ac04f103181 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44aad0f27da7493788209ac04f103181" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 44469 } }
I20260807 18:33:18.434360 14022 raft_consensus.cc:374] T 00000000000000000000000000000000 P 44aad0f27da7493788209ac04f103181 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:33:18.434448 14022 raft_consensus.cc:729] T 00000000000000000000000000000000 P 44aad0f27da7493788209ac04f103181 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 44aad0f27da7493788209ac04f103181, State: Initialized, Role: FOLLOWER
I20260807 18:33:18.434935 14022 consensus_queue.cc:260] T 00000000000000000000000000000000 P 44aad0f27da7493788209ac04f103181 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44aad0f27da7493788209ac04f103181" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 44469 } }
I20260807 18:33:18.435119 14022 raft_consensus.cc:388] T 00000000000000000000000000000000 P 44aad0f27da7493788209ac04f103181 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:33:18.435207 14022 raft_consensus.cc:482] T 00000000000000000000000000000000 P 44aad0f27da7493788209ac04f103181 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:33:18.435314 14022 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 44aad0f27da7493788209ac04f103181 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:33:18.436348 14022 raft_consensus.cc:504] T 00000000000000000000000000000000 P 44aad0f27da7493788209ac04f103181 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44aad0f27da7493788209ac04f103181" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 44469 } }
I20260807 18:33:18.436707 14022 leader_election.cc:302] T 00000000000000000000000000000000 P 44aad0f27da7493788209ac04f103181 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 44aad0f27da7493788209ac04f103181; no voters:
I20260807 18:33:18.437059 14022 leader_election.cc:288] T 00000000000000000000000000000000 P 44aad0f27da7493788209ac04f103181 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:33:18.437201 14027 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 44aad0f27da7493788209ac04f103181 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:33:18.437461 14027 raft_consensus.cc:686] T 00000000000000000000000000000000 P 44aad0f27da7493788209ac04f103181 [term 1 LEADER]: Becoming Leader. State: Replica: 44aad0f27da7493788209ac04f103181, State: Running, Role: LEADER
I20260807 18:33:18.437961 14027 consensus_queue.cc:237] T 00000000000000000000000000000000 P 44aad0f27da7493788209ac04f103181 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44aad0f27da7493788209ac04f103181" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 44469 } }
I20260807 18:33:18.438395 14022 sys_catalog.cc:564] T 00000000000000000000000000000000 P 44aad0f27da7493788209ac04f103181 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:33:18.439873 14028 sys_catalog.cc:455] T 00000000000000000000000000000000 P 44aad0f27da7493788209ac04f103181 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "44aad0f27da7493788209ac04f103181" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44aad0f27da7493788209ac04f103181" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 44469 } } }
I20260807 18:33:18.439881 14029 sys_catalog.cc:455] T 00000000000000000000000000000000 P 44aad0f27da7493788209ac04f103181 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 44aad0f27da7493788209ac04f103181. Latest consensus state: current_term: 1 leader_uuid: "44aad0f27da7493788209ac04f103181" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44aad0f27da7493788209ac04f103181" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 44469 } } }
I20260807 18:33:18.440028 14028 sys_catalog.cc:458] T 00000000000000000000000000000000 P 44aad0f27da7493788209ac04f103181 [sys.catalog]: This master's current role is: LEADER
I20260807 18:33:18.440029 14029 sys_catalog.cc:458] T 00000000000000000000000000000000 P 44aad0f27da7493788209ac04f103181 [sys.catalog]: This master's current role is: LEADER
I20260807 18:33:18.440647 14039 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:33:18.444952 14039 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:33:18.451104 14039 catalog_manager.cc:1347] Generated new cluster ID: a8f850a0a8e644369d24987c514d401f
I20260807 18:33:18.451186 14039 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:33:18.461196 14039 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:33:18.462296 14039 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:33:18.469622 14039 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 44aad0f27da7493788209ac04f103181: Generated new TSK 0
I20260807 18:33:18.470425 14039 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.14.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.14.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.14.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.65.keytab.
Entry for principal kudu/127.11.14.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.65.keytab.
I20260807 18:33:18.511075 11321 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkz56XN/build/debug/bin/kudu
/tmp/dist-test-taskkz56XN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127579900523-11321-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127579900523-11321-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127579900523-11321-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127579900523-11321-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.14.65:0
--local_ip_for_outbound_sockets=127.11.14.65
--webserver_interface=127.11.14.65
--webserver_port=0
--tserver_master_addrs=127.11.14.126:44469
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.65.keytab
--principal=kudu/127.11.14.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.14.84:41087
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127579900523-11321-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127579900523-11321-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127579900523-11321-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:33:18.631345 14048 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:33:18.631752 14048 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:33:18.631840 14048 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:33:18.636088 14048 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:33:18.636159 14048 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:33:18.636222 14048 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.14.65
I20260807 18:33:18.642189 14048 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.14.84:41087
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127579900523-11321-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127579900523-11321-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.65.keytab
--principal=kudu/127.11.14.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.14.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127579900523-11321-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.14.65
--webserver_port=0
--tserver_master_addrs=127.11.14.126:44469
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14048
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.14.65
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127579900523-11321-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:33:18.643412 14048 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:33:18.644577 14048 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:33:18.654829 14053 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:33:18.654868 14054 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:33:18.654867 14048 server_base.cc:1034] running on GCE node
W20260807 18:33:18.655004 14057 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:33:18.655687 14048 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:33:18.656332 14048 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:33:18.657500 14048 hybrid_clock.cc:648] HybridClock initialized: now 1786127598657483 us; error 33 us; skew 500 ppm
Aug 07 18:33:18 dist-test-slave-6k22 krb5kdc[13935](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127598, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:33:18.660492 14048 init.cc:377] Logged in from keytab as kudu/127.11.14.65@KRBTEST.COM (short username kudu)
I20260807 18:33:18.661378 14048 webserver.cc:460] Webserver started at http://127.11.14.65:43749/ using document root <none> and password file <none>
I20260807 18:33:18.661948 14048 fs_manager.cc:359] Metadata directory not provided
I20260807 18:33:18.662011 14048 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:33:18.662179 14048 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:33:18.664333 14048 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127579900523-11321-0/minicluster-data/ts-0/data/instance:
uuid: "16f502cceb6c46d786a6fdfc2ca67376"
format_stamp: "Formatted at 2026-08-07 18:33:18 on dist-test-slave-6k22"
server_key: "01d62aadb6fb9154cb9154782da06f56"
server_key_iv: "1b3f00e7f11c2fa1ae4163d4f9d4c9d6"
server_key_version: "encryptionkey@0"
I20260807 18:33:18.664885 14048 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127579900523-11321-0/minicluster-data/ts-0/wal/instance:
uuid: "16f502cceb6c46d786a6fdfc2ca67376"
format_stamp: "Formatted at 2026-08-07 18:33:18 on dist-test-slave-6k22"
server_key: "01d62aadb6fb9154cb9154782da06f56"
server_key_iv: "1b3f00e7f11c2fa1ae4163d4f9d4c9d6"
server_key_version: "encryptionkey@0"
I20260807 18:33:18.668954 14048 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 18:33:18.671468 14064 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:33:18.673177 14048 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:33:18.673321 14048 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127579900523-11321-0/minicluster-data/ts-0/data,/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127579900523-11321-0/minicluster-data/ts-0/wal
uuid: "16f502cceb6c46d786a6fdfc2ca67376"
format_stamp: "Formatted at 2026-08-07 18:33:18 on dist-test-slave-6k22"
server_key: "01d62aadb6fb9154cb9154782da06f56"
server_key_iv: "1b3f00e7f11c2fa1ae4163d4f9d4c9d6"
server_key_version: "encryptionkey@0"
I20260807 18:33:18.673450 14048 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127579900523-11321-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127579900523-11321-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127579900523-11321-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:33:18.696784 14048 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:33:18.700105 14048 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:33:18.700330 14048 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:33:18.700969 14048 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:33:18.701884 14048 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:33:18.701958 14048 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:33:18.702052 14048 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:33:18.702097 14048 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:33:18.711632 14048 rpc_server.cc:307] RPC server started. Bound to: 127.11.14.65:45689
I20260807 18:33:18.711691 14184 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.14.65:45689 every 8 connection(s)
I20260807 18:33:18.712595 14048 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127579900523-11321-0/minicluster-data/ts-0/data/info.pb
I20260807 18:33:18.719393 11321 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkz56XN/build/debug/bin/kudu as pid 14048
I20260807 18:33:18.719520 11321 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127579900523-11321-0/minicluster-data/ts-0/wal/instance
I20260807 18:33:18.719836 11321 external_mini_cluster.cc:1428] Setting key 2bfc00879cd1bb7ee1bb7e52078a457c
Aug 07 18:33:18 dist-test-slave-6k22 krb5kdc[13935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127598, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.65@KRBTEST.COM for kudu/127.11.14.126@KRBTEST.COM
I20260807 18:33:18.729502 14025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:18.714536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.65:60861 (local address 127.11.14.126:44469)
0807 18:33:18.714731 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:18.714736 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:18.716143 (+ 1407us) server_negotiation.cc:407] Connection header received
0807 18:33:18.717112 (+ 969us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:18.717114 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:18.717162 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:18.717243 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:33:18.718497 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:18.719028 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:18.720693 (+ 1665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:18.721003 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:18.721116 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:18.724808 (+ 3692us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:33:18.724815 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:33:18.724820 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:33:18.724879 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:33:18.726845 (+ 1966us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:18.727616 (+ 771us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:18.727619 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:18.727621 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:18.727708 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:18.728138 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:18.728140 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:18.728141 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:18.728347 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:33:18.728453 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:18.729059 (+ 606us) server_negotiation.cc:299] Negotiation successful
0807 18:33:18.729222 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260807 18:33:18.730003 14187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:18.714854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:44469 (local address 127.11.14.65:60861)
0807 18:33:18.715973 (+ 1119us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:18.716018 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:18.716915 (+ 897us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:18.717407 (+ 492us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:18.717414 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:18.717835 (+ 421us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:33:18.718363 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:18.718378 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:18.719177 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:18.719184 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:18.720549 (+ 1365us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:18.720562 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:18.721185 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:18.721191 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:18.721397 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:18.722058 (+ 661us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:33:18.722083 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:33:18.724554 (+ 2471us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:33:18.726973 (+ 2419us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:33:18.726982 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:33:18.727002 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:33:18.727455 (+ 453us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:33:18.727817 (+ 362us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:33:18.727824 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:33:18.727828 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:33:18.728031 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:33:18.728488 (+ 457us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:33:18.728495 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:33:18.728779 (+ 284us) client_negotiation.cc:769] Sending connection context
0807 18:33:18.728940 (+ 161us) client_negotiation.cc:240] Negotiation successful
0807 18:33:18.729214 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":914,"thread_start_us":766,"threads_started":1}
I20260807 18:33:18.731302 14185 heartbeater.cc:347] Connected to a master server at 127.11.14.126:44469
I20260807 18:33:18.731619 14185 heartbeater.cc:464] Registering TS with master...
I20260807 18:33:18.732288 14185 heartbeater.cc:511] Master 127.11.14.126:44469 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.11.14.66@KRBTEST.COM; defaulting to no policy
I20260807 18:33:18.733923 13984 ts_manager.cc:194] Registered new tserver with Master: 16f502cceb6c46d786a6fdfc2ca67376 (127.11.14.65:45689)
I20260807 18:33:18.735064 13984 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.14.65@KRBTEST.COM'} at 127.11.14.65:60861
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.14.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.14.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.66.keytab.
Entry for principal kudu/127.11.14.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.66.keytab.
I20260807 18:33:18.752983 11321 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkz56XN/build/debug/bin/kudu
/tmp/dist-test-taskkz56XN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127579900523-11321-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127579900523-11321-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127579900523-11321-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127579900523-11321-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.14.66:0
--local_ip_for_outbound_sockets=127.11.14.66
--webserver_interface=127.11.14.66
--webserver_port=0
--tserver_master_addrs=127.11.14.126:44469
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.66.keytab
--principal=kudu/127.11.14.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.14.84:41087
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127579900523-11321-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127579900523-11321-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127579900523-11321-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:33:18.870719 14190 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:33:18.870990 14190 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:33:18.871035 14190 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:33:18.875193 14190 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:33:18.875258 14190 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:33:18.875351 14190 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.14.66
I20260807 18:33:18.880967 14190 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.14.84:41087
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127579900523-11321-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127579900523-11321-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.66.keytab
--principal=kudu/127.11.14.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.14.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127579900523-11321-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.14.66
--webserver_port=0
--tserver_master_addrs=127.11.14.126:44469
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14190
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.14.66
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127579900523-11321-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:33:18.882134 14190 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:33:18.883237 14190 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:33:18.893477 14196 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:33:18.893522 14190 server_base.cc:1034] running on GCE node
W20260807 18:33:18.893486 14197 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:33:18.893477 14199 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:33:18.894182 14190 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:33:18.894824 14190 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:33:18.896011 14190 hybrid_clock.cc:648] HybridClock initialized: now 1786127598895992 us; error 27 us; skew 500 ppm
Aug 07 18:33:18 dist-test-slave-6k22 krb5kdc[13935](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127598, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:33:18.898754 14190 init.cc:377] Logged in from keytab as kudu/127.11.14.66@KRBTEST.COM (short username kudu)
I20260807 18:33:18.899750 14190 webserver.cc:460] Webserver started at http://127.11.14.66:36837/ using document root <none> and password file <none>
I20260807 18:33:18.900348 14190 fs_manager.cc:359] Metadata directory not provided
I20260807 18:33:18.900445 14190 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:33:18.900684 14190 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:33:18.902822 14190 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127579900523-11321-0/minicluster-data/ts-1/data/instance:
uuid: "fd1a297263cf49b1880ae4ebb4e1bebf"
format_stamp: "Formatted at 2026-08-07 18:33:18 on dist-test-slave-6k22"
server_key: "7f715ace5c98b20fd24d4ae42fea7f01"
server_key_iv: "81d3226e9579d4d14b421f349735b1f2"
server_key_version: "encryptionkey@0"
I20260807 18:33:18.903395 14190 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127579900523-11321-0/minicluster-data/ts-1/wal/instance:
uuid: "fd1a297263cf49b1880ae4ebb4e1bebf"
format_stamp: "Formatted at 2026-08-07 18:33:18 on dist-test-slave-6k22"
server_key: "7f715ace5c98b20fd24d4ae42fea7f01"
server_key_iv: "81d3226e9579d4d14b421f349735b1f2"
server_key_version: "encryptionkey@0"
I20260807 18:33:18.907263 14190 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260807 18:33:18.909902 14206 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:33:18.911137 14190 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260807 18:33:18.911294 14190 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127579900523-11321-0/minicluster-data/ts-1/data,/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127579900523-11321-0/minicluster-data/ts-1/wal
uuid: "fd1a297263cf49b1880ae4ebb4e1bebf"
format_stamp: "Formatted at 2026-08-07 18:33:18 on dist-test-slave-6k22"
server_key: "7f715ace5c98b20fd24d4ae42fea7f01"
server_key_iv: "81d3226e9579d4d14b421f349735b1f2"
server_key_version: "encryptionkey@0"
I20260807 18:33:18.911412 14190 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127579900523-11321-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127579900523-11321-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127579900523-11321-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:33:18.922782 14190 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:33:18.925750 14190 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:33:18.925969 14190 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:33:18.926623 14190 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:33:18.927531 14190 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:33:18.927639 14190 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:33:18.927717 14190 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:33:18.927770 14190 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:33:18.937345 14190 rpc_server.cc:307] RPC server started. Bound to: 127.11.14.66:39663
I20260807 18:33:18.937366 14326 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.14.66:39663 every 8 connection(s)
I20260807 18:33:18.938387 14190 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127579900523-11321-0/minicluster-data/ts-1/data/info.pb
I20260807 18:33:18.940321 11321 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkz56XN/build/debug/bin/kudu as pid 14190
I20260807 18:33:18.940435 11321 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127579900523-11321-0/minicluster-data/ts-1/wal/instance
I20260807 18:33:18.940778 11321 external_mini_cluster.cc:1428] Setting key 555b70e476b29825f86760ce05c0552b
Aug 07 18:33:18 dist-test-slave-6k22 krb5kdc[13935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127598, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.66@KRBTEST.COM for kudu/127.11.14.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.11.14.67@KRBTEST.COM; defaulting to no policy
I20260807 18:33:18.953869 14025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:18.940805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.66:54685 (local address 127.11.14.126:44469)
0807 18:33:18.940969 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:18.940974 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:18.941889 (+ 915us) server_negotiation.cc:407] Connection header received
0807 18:33:18.943001 (+ 1112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:18.943004 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:18.943050 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:18.943132 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:33:18.944838 (+ 1706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:18.945688 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:18.946830 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:18.947260 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:18.947409 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:18.949971 (+ 2562us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:33:18.949975 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:33:18.949978 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:33:18.950010 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:33:18.951735 (+ 1725us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:18.952293 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:18.952296 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:18.952297 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:18.952375 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:18.952712 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:18.952715 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:18.952717 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:18.952899 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:33:18.953063 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:18.953639 (+ 576us) server_negotiation.cc:299] Negotiation successful
0807 18:33:18.953744 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260807 18:33:18.954800 14329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:18.940980 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:44469 (local address 127.11.14.66:54685)
0807 18:33:18.941697 (+ 717us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:18.941765 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:18.942737 (+ 972us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:18.943344 (+ 607us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:18.943356 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:18.943939 (+ 583us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:33:18.944668 (+ 729us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:18.944690 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:18.945831 (+ 1141us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:18.945833 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:18.946699 (+ 866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:18.946708 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:18.947420 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:18.947422 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:18.947616 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:18.948307 (+ 691us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:33:18.948325 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:33:18.949792 (+ 1467us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:33:18.951848 (+ 2056us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:33:18.951853 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:33:18.951867 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:33:18.952188 (+ 321us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:33:18.952465 (+ 277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:33:18.952468 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:33:18.952469 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:33:18.952615 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:33:18.953076 (+ 461us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:33:18.953083 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:33:18.953414 (+ 331us) client_negotiation.cc:769] Sending connection context
0807 18:33:18.953585 (+ 171us) client_negotiation.cc:240] Negotiation successful
0807 18:33:18.953867 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":477,"thread_start_us":132,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.14.67@KRBTEST.COM" created.
I20260807 18:33:18.956164 14327 heartbeater.cc:347] Connected to a master server at 127.11.14.126:44469
I20260807 18:33:18.956442 14327 heartbeater.cc:464] Registering TS with master...
I20260807 18:33:18.957099 14327 heartbeater.cc:511] Master 127.11.14.126:44469 requested a full tablet report, sending...
I20260807 18:33:18.958333 13984 ts_manager.cc:194] Registered new tserver with Master: fd1a297263cf49b1880ae4ebb4e1bebf (127.11.14.66:39663)
I20260807 18:33:18.958932 13984 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.14.66@KRBTEST.COM'} at 127.11.14.66:54685
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.14.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.67.keytab.
Entry for principal kudu/127.11.14.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.67.keytab.
I20260807 18:33:18.974160 11321 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkz56XN/build/debug/bin/kudu
/tmp/dist-test-taskkz56XN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127579900523-11321-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127579900523-11321-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127579900523-11321-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127579900523-11321-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.14.67:0
--local_ip_for_outbound_sockets=127.11.14.67
--webserver_interface=127.11.14.67
--webserver_port=0
--tserver_master_addrs=127.11.14.126:44469
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.67.keytab
--principal=kudu/127.11.14.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.14.84:41087
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127579900523-11321-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127579900523-11321-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127579900523-11321-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:33:19.092255 14334 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:33:19.092511 14334 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:33:19.092554 14334 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:33:19.096572 14334 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:33:19.096639 14334 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:33:19.096697 14334 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.14.67
I20260807 18:33:19.101953 14334 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.14.84:41087
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127579900523-11321-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127579900523-11321-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.67.keytab
--principal=kudu/127.11.14.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.14.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127579900523-11321-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.14.67
--webserver_port=0
--tserver_master_addrs=127.11.14.126:44469
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14334
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.14.67
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127579900523-11321-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:33:19.103013 14334 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:33:19.104087 14334 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:33:19.114037 14343 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:33:19.114053 14340 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:33:19.114326 14334 server_base.cc:1034] running on GCE node
W20260807 18:33:19.114037 14341 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:33:19.114885 14334 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:33:19.115613 14334 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:33:19.116814 14334 hybrid_clock.cc:648] HybridClock initialized: now 1786127599116789 us; error 35 us; skew 500 ppm
Aug 07 18:33:19 dist-test-slave-6k22 krb5kdc[13935](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127599, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:33:19.119643 14334 init.cc:377] Logged in from keytab as kudu/127.11.14.67@KRBTEST.COM (short username kudu)
I20260807 18:33:19.120512 14334 webserver.cc:460] Webserver started at http://127.11.14.67:44545/ using document root <none> and password file <none>
I20260807 18:33:19.121093 14334 fs_manager.cc:359] Metadata directory not provided
I20260807 18:33:19.121189 14334 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:33:19.121450 14334 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:33:19.123535 14334 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127579900523-11321-0/minicluster-data/ts-2/data/instance:
uuid: "77a64dab70d2455ea08c5b233ffbd83a"
format_stamp: "Formatted at 2026-08-07 18:33:19 on dist-test-slave-6k22"
server_key: "cdb0c9d114d6303c7fda2dc3a3bfc1cb"
server_key_iv: "f9b1d131d494b8adeb63ef105852c424"
server_key_version: "encryptionkey@0"
I20260807 18:33:19.124177 14334 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127579900523-11321-0/minicluster-data/ts-2/wal/instance:
uuid: "77a64dab70d2455ea08c5b233ffbd83a"
format_stamp: "Formatted at 2026-08-07 18:33:19 on dist-test-slave-6k22"
server_key: "cdb0c9d114d6303c7fda2dc3a3bfc1cb"
server_key_iv: "f9b1d131d494b8adeb63ef105852c424"
server_key_version: "encryptionkey@0"
I20260807 18:33:19.128026 14334 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 18:33:19.130476 14350 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:33:19.131826 14334 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 18:33:19.131969 14334 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127579900523-11321-0/minicluster-data/ts-2/data,/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127579900523-11321-0/minicluster-data/ts-2/wal
uuid: "77a64dab70d2455ea08c5b233ffbd83a"
format_stamp: "Formatted at 2026-08-07 18:33:19 on dist-test-slave-6k22"
server_key: "cdb0c9d114d6303c7fda2dc3a3bfc1cb"
server_key_iv: "f9b1d131d494b8adeb63ef105852c424"
server_key_version: "encryptionkey@0"
I20260807 18:33:19.132083 14334 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127579900523-11321-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127579900523-11321-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127579900523-11321-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:33:19.149163 14334 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:33:19.152242 14334 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:33:19.152446 14334 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:33:19.153039 14334 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:33:19.153916 14334 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:33:19.153988 14334 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:33:19.154065 14334 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:33:19.154109 14334 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:33:19.163349 14334 rpc_server.cc:307] RPC server started. Bound to: 127.11.14.67:45775
I20260807 18:33:19.163383 14472 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.14.67:45775 every 8 connection(s)
I20260807 18:33:19.164295 14334 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127579900523-11321-0/minicluster-data/ts-2/data/info.pb
I20260807 18:33:19.171696 11321 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkz56XN/build/debug/bin/kudu as pid 14334
I20260807 18:33:19.171890 11321 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786127579900523-11321-0/minicluster-data/ts-2/wal/instance
I20260807 18:33:19.172224 11321 external_mini_cluster.cc:1428] Setting key e79ae3fb3efc1a1655f007e98995ebe1
Aug 07 18:33:19 dist-test-slave-6k22 krb5kdc[13935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127599, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.67@KRBTEST.COM for kudu/127.11.14.126@KRBTEST.COM
I20260807 18:33:19.178369 14025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:19.166183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.67:52061 (local address 127.11.14.126:44469)
0807 18:33:19.166332 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:19.166335 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:19.167382 (+ 1047us) server_negotiation.cc:407] Connection header received
0807 18:33:19.168309 (+ 927us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:19.168312 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:19.168369 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:19.168454 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:33:19.169616 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:19.170182 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:19.171464 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:19.171889 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:19.171993 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:19.174762 (+ 2769us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:33:19.174765 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:33:19.174767 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:33:19.174791 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:33:19.176322 (+ 1531us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:19.176843 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:19.176846 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:19.176848 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:19.176928 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:19.177259 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:19.177261 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:19.177262 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:19.177436 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:33:19.177515 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:19.178084 (+ 569us) server_negotiation.cc:299] Negotiation successful
0807 18:33:19.178181 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260807 18:33:19.179198 14476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:19.166572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:44469 (local address 127.11.14.67:52061)
0807 18:33:19.167218 (+ 646us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:19.167270 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:19.168096 (+ 826us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:19.168622 (+ 526us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:19.168630 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:19.168986 (+ 356us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:33:19.169470 (+ 484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:19.169484 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:19.170326 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:19.170335 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:19.171336 (+ 1001us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:19.171347 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:19.172060 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:19.172066 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:19.172313 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:19.173039 (+ 726us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:33:19.173063 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:33:19.174602 (+ 1539us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:33:19.176425 (+ 1823us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:33:19.176430 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:33:19.176441 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:33:19.176731 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:33:19.177016 (+ 285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:33:19.177018 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:33:19.177019 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:33:19.177163 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:33:19.177526 (+ 363us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:33:19.177531 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:33:19.177847 (+ 316us) client_negotiation.cc:769] Sending connection context
0807 18:33:19.178012 (+ 165us) client_negotiation.cc:240] Negotiation successful
0807 18:33:19.178269 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":461,"thread_start_us":115,"threads_started":1}
I20260807 18:33:19.180408 14473 heartbeater.cc:347] Connected to a master server at 127.11.14.126:44469
I20260807 18:33:19.180627 14473 heartbeater.cc:464] Registering TS with master...
I20260807 18:33:19.181092 14473 heartbeater.cc:511] Master 127.11.14.126:44469 requested a full tablet report, sending...
I20260807 18:33:19.182220 13984 ts_manager.cc:194] Registered new tserver with Master: 77a64dab70d2455ea08c5b233ffbd83a (127.11.14.67:45775)
I20260807 18:33:19.182775 13984 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.14.67@KRBTEST.COM'} at 127.11.14.67:52061
I20260807 18:33:19.186174 11321 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260807 18:33:19.205698 14025 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:33:19.204089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35022 (local address 127.11.14.126:44469)
0807 18:33:19.204268 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:19.204273 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:19.204425 (+ 152us) server_negotiation.cc:407] Connection header received
0807 18:33:19.204599 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:19.204603 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:19.204665 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:19.204762 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:33:19.205562 (+ 800us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35022: BlockingRecv error: recv got EOF from 127.0.0.1:35022 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
I20260807 18:33:19.206744 11321 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkz56XN/build/debug/bin/kudu with pid 14048
I20260807 18:33:19.215976 11321 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkz56XN/build/debug/bin/kudu with pid 14190
I20260807 18:33:19.224344 11321 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkz56XN/build/debug/bin/kudu with pid 14334
I20260807 18:33:19.235747 11321 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkz56XN/build/debug/bin/kudu with pid 13950
2026-08-07T18:33:19Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (2293 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786127579900523-11321-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:33:19 dist-test-slave-6k22 krb5kdc[14486](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:33:19 dist-test-slave-6k22 krb5kdc[14486](info): set up 2 sockets
Aug 07 18:33:19 dist-test-slave-6k22 krb5kdc[14486](info): commencing operation
krb5kdc: starting...
W20260807 18:33:20.351186 11321 mini_kdc.cc:121] Time spent starting KDC: real 1.090s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786127579900523-11321-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786127579900523-11321-0/krb5kdc/test-user.keytab.
Aug 07 18:33:20 dist-test-slave-6k22 krb5kdc[14486](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127600, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:33:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:33:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.14.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.14.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.14.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.126.keytab.
Entry for principal kudu/127.11.14.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.126.keytab.
I20260807 18:33:20.487367 11321 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkz56XN/build/debug/bin/kudu
/tmp/dist-test-taskkz56XN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786127579900523-11321-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786127579900523-11321-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786127579900523-11321-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786127579900523-11321-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.14.126:32867
--webserver_interface=127.11.14.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.126.keytab
--principal=kudu/127.11.14.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.14.84:43273
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.14.126:32867
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786127579900523-11321-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786127579900523-11321-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786127579900523-11321-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:33:20.602985 14499 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:33:20.603307 14499 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:33:20.603394 14499 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:33:20.607477 14499 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:33:20.607599 14499 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:33:20.607657 14499 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:33:20.607717 14499 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:33:20.607753 14499 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:33:20.613318 14499 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.14.84:43273
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786127579900523-11321-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786127579900523-11321-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.14.126:32867
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.126.keytab
--principal=kudu/127.11.14.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.14.126:32867
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786127579900523-11321-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.14.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14499
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786127579900523-11321-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:33:20.614512 14499 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:33:20.615658 14499 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:33:20.624614 14506 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:33:20.624718 14499 server_base.cc:1034] running on GCE node
W20260807 18:33:20.624603 14508 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:33:20.624619 14505 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:33:20.625438 14499 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:33:20.626622 14499 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:33:20.627826 14499 hybrid_clock.cc:648] HybridClock initialized: now 1786127600627811 us; error 26 us; skew 500 ppm
Aug 07 18:33:20 dist-test-slave-6k22 krb5kdc[14486](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127600, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:33:20.630874 14499 init.cc:377] Logged in from keytab as kudu/127.11.14.126@KRBTEST.COM (short username kudu)
I20260807 18:33:20.631896 14499 webserver.cc:460] Webserver started at http://127.11.14.126:41307/ using document root <none> and password file <none>
I20260807 18:33:20.632514 14499 fs_manager.cc:359] Metadata directory not provided
I20260807 18:33:20.632611 14499 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:33:20.632858 14499 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:33:20.634976 14499 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786127579900523-11321-0/minicluster-data/master-0/data/instance:
uuid: "188c4c927c35474e97c45cbe0a72121a"
format_stamp: "Formatted at 2026-08-07 18:33:20 on dist-test-slave-6k22"
server_key: "8ad78ff47d63ccb9b85efd1bc63d0371"
server_key_iv: "060c2eb381c07e8003a4428abf0838d2"
server_key_version: "encryptionkey@0"
I20260807 18:33:20.635638 14499 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786127579900523-11321-0/minicluster-data/master-0/wal/instance:
uuid: "188c4c927c35474e97c45cbe0a72121a"
format_stamp: "Formatted at 2026-08-07 18:33:20 on dist-test-slave-6k22"
server_key: "8ad78ff47d63ccb9b85efd1bc63d0371"
server_key_iv: "060c2eb381c07e8003a4428abf0838d2"
server_key_version: "encryptionkey@0"
I20260807 18:33:20.639482 14499 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260807 18:33:20.642110 14516 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:33:20.643527 14499 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 18:33:20.643736 14499 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786127579900523-11321-0/minicluster-data/master-0/data,/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786127579900523-11321-0/minicluster-data/master-0/wal
uuid: "188c4c927c35474e97c45cbe0a72121a"
format_stamp: "Formatted at 2026-08-07 18:33:20 on dist-test-slave-6k22"
server_key: "8ad78ff47d63ccb9b85efd1bc63d0371"
server_key_iv: "060c2eb381c07e8003a4428abf0838d2"
server_key_version: "encryptionkey@0"
I20260807 18:33:20.643865 14499 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786127579900523-11321-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786127579900523-11321-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786127579900523-11321-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:33:20.663803 14499 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:33:20.667241 14499 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:33:20.667470 14499 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:33:20.675275 14499 rpc_server.cc:307] RPC server started. Bound to: 127.11.14.126:32867
I20260807 18:33:20.675801 14571 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.14.126:32867 every 8 connection(s)
I20260807 18:33:20.676445 14499 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786127579900523-11321-0/minicluster-data/master-0/data/info.pb
I20260807 18:33:20.679550 14573 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:33:20.685371 14573 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 188c4c927c35474e97c45cbe0a72121a: Bootstrap starting.
I20260807 18:33:20.685748 11321 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkz56XN/build/debug/bin/kudu as pid 14499
I20260807 18:33:20.685906 11321 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786127579900523-11321-0/minicluster-data/master-0/wal/instance
I20260807 18:33:20.686203 11321 external_mini_cluster.cc:1428] Setting key a0fda5de5749e6939274d731ec17295b
I20260807 18:33:20.688659 14573 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 188c4c927c35474e97c45cbe0a72121a: Neither blocks nor log segments found. Creating new log.
I20260807 18:33:20.689597 14573 log.cc:824] T 00000000000000000000000000000000 P 188c4c927c35474e97c45cbe0a72121a: Log is configured to *not* fsync() on all Append() calls
I20260807 18:33:20.691962 14573 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 188c4c927c35474e97c45cbe0a72121a: No bootstrap required, opened a new log
I20260807 18:33:20.696389 14573 raft_consensus.cc:348] T 00000000000000000000000000000000 P 188c4c927c35474e97c45cbe0a72121a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "188c4c927c35474e97c45cbe0a72121a" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 32867 } }
I20260807 18:33:20.696720 14573 raft_consensus.cc:374] T 00000000000000000000000000000000 P 188c4c927c35474e97c45cbe0a72121a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:33:20.696776 14573 raft_consensus.cc:729] T 00000000000000000000000000000000 P 188c4c927c35474e97c45cbe0a72121a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 188c4c927c35474e97c45cbe0a72121a, State: Initialized, Role: FOLLOWER
Aug 07 18:33:20 dist-test-slave-6k22 krb5kdc[14486](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127600, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.14.126@KRBTEST.COM
I20260807 18:33:20.697355 14573 consensus_queue.cc:260] T 00000000000000000000000000000000 P 188c4c927c35474e97c45cbe0a72121a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "188c4c927c35474e97c45cbe0a72121a" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 32867 } }
I20260807 18:33:20.697539 14573 raft_consensus.cc:388] T 00000000000000000000000000000000 P 188c4c927c35474e97c45cbe0a72121a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:33:20.697628 14573 raft_consensus.cc:482] T 00000000000000000000000000000000 P 188c4c927c35474e97c45cbe0a72121a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:33:20.697732 14573 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 188c4c927c35474e97c45cbe0a72121a [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:33:20.698990 14573 raft_consensus.cc:504] T 00000000000000000000000000000000 P 188c4c927c35474e97c45cbe0a72121a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "188c4c927c35474e97c45cbe0a72121a" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 32867 } }
I20260807 18:33:20.699362 14573 leader_election.cc:302] T 00000000000000000000000000000000 P 188c4c927c35474e97c45cbe0a72121a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 188c4c927c35474e97c45cbe0a72121a; no voters:
I20260807 18:33:20.699762 14573 leader_election.cc:288] T 00000000000000000000000000000000 P 188c4c927c35474e97c45cbe0a72121a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:33:20.699856 14579 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 188c4c927c35474e97c45cbe0a72121a [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:33:20.700075 14579 raft_consensus.cc:686] T 00000000000000000000000000000000 P 188c4c927c35474e97c45cbe0a72121a [term 1 LEADER]: Becoming Leader. State: Replica: 188c4c927c35474e97c45cbe0a72121a, State: Running, Role: LEADER
I20260807 18:33:20.700438 14579 consensus_queue.cc:237] T 00000000000000000000000000000000 P 188c4c927c35474e97c45cbe0a72121a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "188c4c927c35474e97c45cbe0a72121a" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 32867 } }
I20260807 18:33:20.701211 14573 sys_catalog.cc:564] T 00000000000000000000000000000000 P 188c4c927c35474e97c45cbe0a72121a [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:33:20.702409 14580 sys_catalog.cc:455] T 00000000000000000000000000000000 P 188c4c927c35474e97c45cbe0a72121a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 188c4c927c35474e97c45cbe0a72121a. Latest consensus state: current_term: 1 leader_uuid: "188c4c927c35474e97c45cbe0a72121a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "188c4c927c35474e97c45cbe0a72121a" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 32867 } } }
I20260807 18:33:20.702412 14581 sys_catalog.cc:455] T 00000000000000000000000000000000 P 188c4c927c35474e97c45cbe0a72121a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "188c4c927c35474e97c45cbe0a72121a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "188c4c927c35474e97c45cbe0a72121a" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 32867 } } }
I20260807 18:33:20.702581 14581 sys_catalog.cc:458] T 00000000000000000000000000000000 P 188c4c927c35474e97c45cbe0a72121a [sys.catalog]: This master's current role is: LEADER
I20260807 18:33:20.702580 14580 sys_catalog.cc:458] T 00000000000000000000000000000000 P 188c4c927c35474e97c45cbe0a72121a [sys.catalog]: This master's current role is: LEADER
I20260807 18:33:20.704281 14577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:20.687745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56608 (local address 127.11.14.126:32867)
0807 18:33:20.688232 (+ 487us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:20.688261 (+ 29us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:20.688305 (+ 44us) server_negotiation.cc:407] Connection header received
0807 18:33:20.689219 (+ 914us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:20.689244 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:20.689718 (+ 474us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:20.690102 (+ 384us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:33:20.691010 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:20.692493 (+ 1483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:20.693484 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:20.694212 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:20.694375 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:20.697656 (+ 3281us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:33:20.697681 (+ 25us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:33:20.697706 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:33:20.697746 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:33:20.699942 (+ 2196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:20.700508 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:20.700515 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:20.700521 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:20.700675 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:20.701017 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:20.701022 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:20.701024 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:20.701619 (+ 595us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:33:20.701861 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:20.702822 (+ 961us) server_negotiation.cc:299] Negotiation successful
0807 18:33:20.703176 (+ 354us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":317,"thread_start_us":120,"threads_started":1}
W20260807 18:33:20.706549 14595 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 188c4c927c35474e97c45cbe0a72121a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 18:33:20.706641 14595 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 18:33:20.706709 14589 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:33:20.707615 14589 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:33:20.707737 11321 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 18:33:20.714016 14589 catalog_manager.cc:1347] Generated new cluster ID: 5aef75c4c6ba477187d55810988f13e2
I20260807 18:33:20.714084 14589 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:33:20.728677 14577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:20.722917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56622 (local address 127.11.14.126:32867)
0807 18:33:20.723089 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:20.723095 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:20.723402 (+ 307us) server_negotiation.cc:407] Connection header received
0807 18:33:20.723650 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:20.723653 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:20.723704 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:20.723804 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:33:20.724414 (+ 610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:20.724955 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:20.725871 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:20.726140 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:20.726269 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:20.727284 (+ 1015us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:33:20.727296 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:33:20.727299 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:33:20.728076 (+ 777us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:33:20.728126 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:33:20.728148 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:33:20.728274 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:20.728401 (+ 127us) server_negotiation.cc:299] Negotiation successful
0807 18:33:20.728484 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260807 18:33:20.735586 14589 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:33:20.736927 14589 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:33:20.746577 14589 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 188c4c927c35474e97c45cbe0a72121a: Generated new TSK 0
I20260807 18:33:20.747514 14589 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:33:20.753644 11321 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkz56XN/build/debug/bin/kudu with pid 14499
2026-08-07T18:33:20Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1516 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127579900523-11321-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:33:20 dist-test-slave-6k22 krb5kdc[14611](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:33:20 dist-test-slave-6k22 krb5kdc[14611](info): set up 2 sockets
Aug 07 18:33:20 dist-test-slave-6k22 krb5kdc[14611](info): commencing operation
krb5kdc: starting...
W20260807 18:33:21.859297 11321 mini_kdc.cc:121] Time spent starting KDC: real 1.081s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127579900523-11321-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127579900523-11321-0/krb5kdc/test-user.keytab.
Aug 07 18:33:21 dist-test-slave-6k22 krb5kdc[14611](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127601, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:33:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:33:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.14.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.14.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.14.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.126.keytab.
Entry for principal kudu/127.11.14.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.126.keytab.
I20260807 18:33:21.992936 11321 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkz56XN/build/debug/bin/kudu
/tmp/dist-test-taskkz56XN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127579900523-11321-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127579900523-11321-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127579900523-11321-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127579900523-11321-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.14.126:44795
--webserver_interface=127.11.14.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.126.keytab
--principal=kudu/127.11.14.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.14.84:40503
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.14.126:44795
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127579900523-11321-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127579900523-11321-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127579900523-11321-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:33:22.108907 14627 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:33:22.109222 14627 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:33:22.109305 14627 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:33:22.113361 14627 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:33:22.113444 14627 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:33:22.113482 14627 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:33:22.113539 14627 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:33:22.113575 14627 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:33:22.118919 14627 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.14.84:40503
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127579900523-11321-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127579900523-11321-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.14.126:44795
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.126.keytab
--principal=kudu/127.11.14.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.14.126:44795
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127579900523-11321-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.14.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14627
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127579900523-11321-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:33:22.120107 14627 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:33:22.121125 14627 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:33:22.129913 14635 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:33:22.129913 14633 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:33:22.130014 14632 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:33:22.130316 14627 server_base.cc:1034] running on GCE node
I20260807 18:33:22.130874 14627 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:33:22.132117 14627 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:33:22.133316 14627 hybrid_clock.cc:648] HybridClock initialized: now 1786127602133300 us; error 27 us; skew 500 ppm
Aug 07 18:33:22 dist-test-slave-6k22 krb5kdc[14611](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127602, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:33:22.142985 14627 init.cc:377] Logged in from keytab as kudu/127.11.14.126@KRBTEST.COM (short username kudu)
I20260807 18:33:22.144084 14627 webserver.cc:460] Webserver started at http://127.11.14.126:39983/ using document root <none> and password file <none>
I20260807 18:33:22.144637 14627 fs_manager.cc:359] Metadata directory not provided
I20260807 18:33:22.144706 14627 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:33:22.144882 14627 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:33:22.146962 14627 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127579900523-11321-0/minicluster-data/master-0/data/instance:
uuid: "fd32a3077b634c29b90b15507d10f158"
format_stamp: "Formatted at 2026-08-07 18:33:22 on dist-test-slave-6k22"
server_key: "1ebe388f10dec3628521ec6f4a086414"
server_key_iv: "e3e4df17155138caa886ad09c69372de"
server_key_version: "encryptionkey@0"
I20260807 18:33:22.147511 14627 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127579900523-11321-0/minicluster-data/master-0/wal/instance:
uuid: "fd32a3077b634c29b90b15507d10f158"
format_stamp: "Formatted at 2026-08-07 18:33:22 on dist-test-slave-6k22"
server_key: "1ebe388f10dec3628521ec6f4a086414"
server_key_iv: "e3e4df17155138caa886ad09c69372de"
server_key_version: "encryptionkey@0"
I20260807 18:33:22.151618 14627 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260807 18:33:22.154202 14642 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:33:22.155516 14627 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 18:33:22.155707 14627 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127579900523-11321-0/minicluster-data/master-0/data,/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127579900523-11321-0/minicluster-data/master-0/wal
uuid: "fd32a3077b634c29b90b15507d10f158"
format_stamp: "Formatted at 2026-08-07 18:33:22 on dist-test-slave-6k22"
server_key: "1ebe388f10dec3628521ec6f4a086414"
server_key_iv: "e3e4df17155138caa886ad09c69372de"
server_key_version: "encryptionkey@0"
I20260807 18:33:22.155808 14627 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127579900523-11321-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127579900523-11321-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127579900523-11321-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:33:22.194371 14627 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:33:22.197642 14627 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:33:22.197813 14627 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:33:22.205089 14627 rpc_server.cc:307] RPC server started. Bound to: 127.11.14.126:44795
I20260807 18:33:22.205440 14700 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.14.126:44795 every 8 connection(s)
I20260807 18:33:22.206144 14627 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127579900523-11321-0/minicluster-data/master-0/data/info.pb
I20260807 18:33:22.209172 14701 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:33:22.214774 14701 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fd32a3077b634c29b90b15507d10f158: Bootstrap starting.
I20260807 18:33:22.215158 11321 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkz56XN/build/debug/bin/kudu as pid 14627
I20260807 18:33:22.215291 11321 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127579900523-11321-0/minicluster-data/master-0/wal/instance
I20260807 18:33:22.215633 11321 external_mini_cluster.cc:1428] Setting key 349412a53af4e948af0bc64560224e3e
I20260807 18:33:22.217767 14701 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fd32a3077b634c29b90b15507d10f158: Neither blocks nor log segments found. Creating new log.
I20260807 18:33:22.218708 14701 log.cc:824] T 00000000000000000000000000000000 P fd32a3077b634c29b90b15507d10f158: Log is configured to *not* fsync() on all Append() calls
I20260807 18:33:22.220820 14701 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fd32a3077b634c29b90b15507d10f158: No bootstrap required, opened a new log
I20260807 18:33:22.224280 14701 raft_consensus.cc:348] T 00000000000000000000000000000000 P fd32a3077b634c29b90b15507d10f158 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd32a3077b634c29b90b15507d10f158" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 44795 } }
I20260807 18:33:22.224539 14701 raft_consensus.cc:374] T 00000000000000000000000000000000 P fd32a3077b634c29b90b15507d10f158 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:33:22.224620 14701 raft_consensus.cc:729] T 00000000000000000000000000000000 P fd32a3077b634c29b90b15507d10f158 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd32a3077b634c29b90b15507d10f158, State: Initialized, Role: FOLLOWER
Aug 07 18:33:22 dist-test-slave-6k22 krb5kdc[14611](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127601, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.14.126@KRBTEST.COM
I20260807 18:33:22.225124 14701 consensus_queue.cc:260] T 00000000000000000000000000000000 P fd32a3077b634c29b90b15507d10f158 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd32a3077b634c29b90b15507d10f158" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 44795 } }
I20260807 18:33:22.225312 14701 raft_consensus.cc:388] T 00000000000000000000000000000000 P fd32a3077b634c29b90b15507d10f158 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:33:22.225391 14701 raft_consensus.cc:482] T 00000000000000000000000000000000 P fd32a3077b634c29b90b15507d10f158 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:33:22.225497 14701 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fd32a3077b634c29b90b15507d10f158 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:33:22.226418 14701 raft_consensus.cc:504] T 00000000000000000000000000000000 P fd32a3077b634c29b90b15507d10f158 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd32a3077b634c29b90b15507d10f158" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 44795 } }
I20260807 18:33:22.226814 14701 leader_election.cc:302] T 00000000000000000000000000000000 P fd32a3077b634c29b90b15507d10f158 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fd32a3077b634c29b90b15507d10f158; no voters:
I20260807 18:33:22.227131 14701 leader_election.cc:288] T 00000000000000000000000000000000 P fd32a3077b634c29b90b15507d10f158 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:33:22.227277 14706 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fd32a3077b634c29b90b15507d10f158 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:33:22.227511 14706 raft_consensus.cc:686] T 00000000000000000000000000000000 P fd32a3077b634c29b90b15507d10f158 [term 1 LEADER]: Becoming Leader. State: Replica: fd32a3077b634c29b90b15507d10f158, State: Running, Role: LEADER
I20260807 18:33:22.227888 14706 consensus_queue.cc:237] T 00000000000000000000000000000000 P fd32a3077b634c29b90b15507d10f158 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd32a3077b634c29b90b15507d10f158" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 44795 } }
I20260807 18:33:22.228475 14701 sys_catalog.cc:564] T 00000000000000000000000000000000 P fd32a3077b634c29b90b15507d10f158 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:33:22.229795 14707 sys_catalog.cc:455] T 00000000000000000000000000000000 P fd32a3077b634c29b90b15507d10f158 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fd32a3077b634c29b90b15507d10f158" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd32a3077b634c29b90b15507d10f158" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 44795 } } }
I20260807 18:33:22.230086 14708 sys_catalog.cc:455] T 00000000000000000000000000000000 P fd32a3077b634c29b90b15507d10f158 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fd32a3077b634c29b90b15507d10f158. Latest consensus state: current_term: 1 leader_uuid: "fd32a3077b634c29b90b15507d10f158" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd32a3077b634c29b90b15507d10f158" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 44795 } } }
I20260807 18:33:22.230403 14708 sys_catalog.cc:458] T 00000000000000000000000000000000 P fd32a3077b634c29b90b15507d10f158 [sys.catalog]: This master's current role is: LEADER
I20260807 18:33:22.230489 14707 sys_catalog.cc:458] T 00000000000000000000000000000000 P fd32a3077b634c29b90b15507d10f158 [sys.catalog]: This master's current role is: LEADER
I20260807 18:33:22.231426 14704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:22.217053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54826 (local address 127.11.14.126:44795)
0807 18:33:22.217506 (+ 453us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:22.217528 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:22.217571 (+ 43us) server_negotiation.cc:407] Connection header received
0807 18:33:22.218462 (+ 891us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:22.218491 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:22.218943 (+ 452us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:22.219378 (+ 435us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:33:22.220239 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:22.221290 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:22.222138 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:22.222620 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:22.222802 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:22.225657 (+ 2855us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:33:22.225664 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:33:22.225687 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:33:22.225715 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:33:22.227904 (+ 2189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:22.228451 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:22.228456 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:22.228463 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:22.228638 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:22.228972 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:22.228976 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:22.228978 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:22.229563 (+ 585us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:33:22.229772 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:22.230038 (+ 266us) server_negotiation.cc:299] Negotiation successful
0807 18:33:22.230358 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":306,"thread_start_us":154,"threads_started":1}
W20260807 18:33:22.233824 14722 catalog_manager.cc:1558] T 00000000000000000000000000000000 P fd32a3077b634c29b90b15507d10f158: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 18:33:22.233939 14722 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 18:33:22.234254 14720 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:33:22.235116 14720 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:33:22.240445 14720 catalog_manager.cc:1347] Generated new cluster ID: 1a32c8a0b7b74b8897b71e4b29757e53
I20260807 18:33:22.240521 14720 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:33:22.278578 14720 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:33:22.279588 14720 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:33:22.285897 14720 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fd32a3077b634c29b90b15507d10f158: Generated new TSK 0
I20260807 18:33:22.286648 14720 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.14.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.14.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.14.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.65.keytab.
Entry for principal kudu/127.11.14.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.65.keytab.
I20260807 18:33:22.319089 11321 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkz56XN/build/debug/bin/kudu
/tmp/dist-test-taskkz56XN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127579900523-11321-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127579900523-11321-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127579900523-11321-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127579900523-11321-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.14.65:0
--local_ip_for_outbound_sockets=127.11.14.65
--webserver_interface=127.11.14.65
--webserver_port=0
--tserver_master_addrs=127.11.14.126:44795
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.65.keytab
--principal=kudu/127.11.14.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.14.84:40503
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127579900523-11321-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127579900523-11321-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127579900523-11321-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:33:22.442050 14728 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:33:22.442344 14728 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:33:22.442391 14728 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:33:22.446476 14728 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:33:22.446547 14728 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:33:22.446606 14728 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.14.65
I20260807 18:33:22.451932 14728 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.14.84:40503
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127579900523-11321-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127579900523-11321-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.65.keytab
--principal=kudu/127.11.14.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.14.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127579900523-11321-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.14.65
--webserver_port=0
--tserver_master_addrs=127.11.14.126:44795
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14728
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.14.65
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127579900523-11321-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:33:22.452986 14728 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:33:22.453979 14728 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:33:22.463929 14734 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:33:22.464125 14728 server_base.cc:1034] running on GCE node
W20260807 18:33:22.463936 14733 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:33:22.463936 14736 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:33:22.464677 14728 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:33:22.465346 14728 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:33:22.466521 14728 hybrid_clock.cc:648] HybridClock initialized: now 1786127602466482 us; error 47 us; skew 500 ppm
Aug 07 18:33:22 dist-test-slave-6k22 krb5kdc[14611](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127602, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:33:22.469405 14728 init.cc:377] Logged in from keytab as kudu/127.11.14.65@KRBTEST.COM (short username kudu)
I20260807 18:33:22.470299 14728 webserver.cc:460] Webserver started at http://127.11.14.65:33357/ using document root <none> and password file <none>
I20260807 18:33:22.470830 14728 fs_manager.cc:359] Metadata directory not provided
I20260807 18:33:22.470891 14728 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:33:22.471135 14728 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:33:22.473209 14728 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127579900523-11321-0/minicluster-data/ts-0/data/instance:
uuid: "18f84f2620bb433aaa1243b07f057379"
format_stamp: "Formatted at 2026-08-07 18:33:22 on dist-test-slave-6k22"
server_key: "ee060adf18b8a3c12590ce963d882646"
server_key_iv: "4004b222bd7881bf1633fcf14f2639e2"
server_key_version: "encryptionkey@0"
I20260807 18:33:22.473731 14728 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127579900523-11321-0/minicluster-data/ts-0/wal/instance:
uuid: "18f84f2620bb433aaa1243b07f057379"
format_stamp: "Formatted at 2026-08-07 18:33:22 on dist-test-slave-6k22"
server_key: "ee060adf18b8a3c12590ce963d882646"
server_key_iv: "4004b222bd7881bf1633fcf14f2639e2"
server_key_version: "encryptionkey@0"
I20260807 18:33:22.477530 14728 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 18:33:22.480088 14743 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:33:22.481604 14728 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:33:22.481720 14728 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127579900523-11321-0/minicluster-data/ts-0/data,/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127579900523-11321-0/minicluster-data/ts-0/wal
uuid: "18f84f2620bb433aaa1243b07f057379"
format_stamp: "Formatted at 2026-08-07 18:33:22 on dist-test-slave-6k22"
server_key: "ee060adf18b8a3c12590ce963d882646"
server_key_iv: "4004b222bd7881bf1633fcf14f2639e2"
server_key_version: "encryptionkey@0"
I20260807 18:33:22.481832 14728 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127579900523-11321-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127579900523-11321-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127579900523-11321-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:33:22.497186 14728 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:33:22.500666 14728 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:33:22.500829 14728 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:33:22.501428 14728 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:33:22.502245 14728 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:33:22.502293 14728 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:33:22.502334 14728 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:33:22.502350 14728 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:33:22.511220 14728 rpc_server.cc:307] RPC server started. Bound to: 127.11.14.65:38141
I20260807 18:33:22.511251 14865 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.14.65:38141 every 8 connection(s)
I20260807 18:33:22.512164 14728 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127579900523-11321-0/minicluster-data/ts-0/data/info.pb
I20260807 18:33:22.517213 11321 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkz56XN/build/debug/bin/kudu as pid 14728
I20260807 18:33:22.517336 11321 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127579900523-11321-0/minicluster-data/ts-0/wal/instance
I20260807 18:33:22.517673 11321 external_mini_cluster.cc:1428] Setting key c42c20f5329289eb0fbae4bc17a20c6c
Aug 07 18:33:22 dist-test-slave-6k22 krb5kdc[14611](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127602, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.65@KRBTEST.COM for kudu/127.11.14.126@KRBTEST.COM
I20260807 18:33:22.527516 14704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:22.514223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.65:43881 (local address 127.11.14.126:44795)
0807 18:33:22.514375 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:22.514378 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:22.515230 (+ 852us) server_negotiation.cc:407] Connection header received
0807 18:33:22.516785 (+ 1555us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:22.516788 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:22.516849 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:22.516978 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:33:22.518492 (+ 1514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:22.519052 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:22.520283 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:22.520708 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:22.520837 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:22.523279 (+ 2442us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:33:22.523286 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:33:22.523292 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:33:22.523326 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:33:22.525269 (+ 1943us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:22.525792 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:22.525797 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:22.525801 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:22.525904 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:22.526246 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:22.526251 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:22.526254 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:22.526504 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:33:22.526617 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:22.527176 (+ 559us) server_negotiation.cc:299] Negotiation successful
0807 18:33:22.527318 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260807 18:33:22.528373 14868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:22.514602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:44795 (local address 127.11.14.65:43881)
0807 18:33:22.515059 (+ 457us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:22.515104 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:22.516513 (+ 1409us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:22.517168 (+ 655us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:22.517176 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:22.517650 (+ 474us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:33:22.518321 (+ 671us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:22.518340 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:22.519208 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:22.519211 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:22.520152 (+ 941us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:22.520162 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:22.520863 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:22.520866 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:22.521046 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:22.521667 (+ 621us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:33:22.521683 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:33:22.523109 (+ 1426us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:33:22.525411 (+ 2302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:33:22.525416 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:33:22.525427 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:33:22.525687 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:33:22.526013 (+ 326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:33:22.526015 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:33:22.526016 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:33:22.526146 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:33:22.526629 (+ 483us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:33:22.526636 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:33:22.526912 (+ 276us) client_negotiation.cc:769] Sending connection context
0807 18:33:22.527074 (+ 162us) client_negotiation.cc:240] Negotiation successful
0807 18:33:22.527338 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":110,"threads_started":1}
WARNING: no policy specified for kudu/127.11.14.66@KRBTEST.COM; defaulting to no policy
I20260807 18:33:22.529767 14866 heartbeater.cc:347] Connected to a master server at 127.11.14.126:44795
I20260807 18:33:22.530055 14866 heartbeater.cc:464] Registering TS with master...
I20260807 18:33:22.530766 14866 heartbeater.cc:511] Master 127.11.14.126:44795 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.14.66@KRBTEST.COM" created.
I20260807 18:33:22.532625 14659 ts_manager.cc:194] Registered new tserver with Master: 18f84f2620bb433aaa1243b07f057379 (127.11.14.65:38141)
I20260807 18:33:22.534122 14659 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.14.65@KRBTEST.COM'} at 127.11.14.65:43881
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.14.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.66.keytab.
Entry for principal kudu/127.11.14.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.66.keytab.
I20260807 18:33:22.548552 11321 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkz56XN/build/debug/bin/kudu
/tmp/dist-test-taskkz56XN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127579900523-11321-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127579900523-11321-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127579900523-11321-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127579900523-11321-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.14.66:0
--local_ip_for_outbound_sockets=127.11.14.66
--webserver_interface=127.11.14.66
--webserver_port=0
--tserver_master_addrs=127.11.14.126:44795
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.66.keytab
--principal=kudu/127.11.14.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.14.84:40503
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127579900523-11321-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127579900523-11321-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127579900523-11321-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:33:22.667032 14871 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:33:22.667357 14871 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:33:22.667456 14871 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:33:22.671548 14871 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:33:22.671705 14871 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:33:22.671813 14871 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.14.66
I20260807 18:33:22.677342 14871 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.14.84:40503
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127579900523-11321-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127579900523-11321-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.66.keytab
--principal=kudu/127.11.14.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.14.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127579900523-11321-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.14.66
--webserver_port=0
--tserver_master_addrs=127.11.14.126:44795
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14871
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.14.66
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127579900523-11321-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:33:22.678486 14871 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:33:22.679540 14871 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:33:22.689957 14877 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:33:22.689957 14876 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:33:22.689980 14879 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:33:22.690215 14871 server_base.cc:1034] running on GCE node
I20260807 18:33:22.690735 14871 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:33:22.691510 14871 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:33:22.692745 14871 hybrid_clock.cc:648] HybridClock initialized: now 1786127602692723 us; error 30 us; skew 500 ppm
Aug 07 18:33:22 dist-test-slave-6k22 krb5kdc[14611](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127602, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:33:22.695827 14871 init.cc:377] Logged in from keytab as kudu/127.11.14.66@KRBTEST.COM (short username kudu)
I20260807 18:33:22.696770 14871 webserver.cc:460] Webserver started at http://127.11.14.66:41957/ using document root <none> and password file <none>
I20260807 18:33:22.697371 14871 fs_manager.cc:359] Metadata directory not provided
I20260807 18:33:22.697474 14871 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:33:22.697721 14871 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:33:22.699885 14871 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127579900523-11321-0/minicluster-data/ts-1/data/instance:
uuid: "663ccd8f5faf48f9b14da93df82ad0e0"
format_stamp: "Formatted at 2026-08-07 18:33:22 on dist-test-slave-6k22"
server_key: "91186dba293fb20e5750245e15d40fde"
server_key_iv: "506ffab88f79e0afc956ac70acae5997"
server_key_version: "encryptionkey@0"
I20260807 18:33:22.700538 14871 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127579900523-11321-0/minicluster-data/ts-1/wal/instance:
uuid: "663ccd8f5faf48f9b14da93df82ad0e0"
format_stamp: "Formatted at 2026-08-07 18:33:22 on dist-test-slave-6k22"
server_key: "91186dba293fb20e5750245e15d40fde"
server_key_iv: "506ffab88f79e0afc956ac70acae5997"
server_key_version: "encryptionkey@0"
I20260807 18:33:22.704519 14871 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260807 18:33:22.707115 14886 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:33:22.708448 14871 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260807 18:33:22.708603 14871 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127579900523-11321-0/minicluster-data/ts-1/data,/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127579900523-11321-0/minicluster-data/ts-1/wal
uuid: "663ccd8f5faf48f9b14da93df82ad0e0"
format_stamp: "Formatted at 2026-08-07 18:33:22 on dist-test-slave-6k22"
server_key: "91186dba293fb20e5750245e15d40fde"
server_key_iv: "506ffab88f79e0afc956ac70acae5997"
server_key_version: "encryptionkey@0"
I20260807 18:33:22.708722 14871 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127579900523-11321-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127579900523-11321-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127579900523-11321-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:33:22.729496 14871 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:33:22.733510 14871 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:33:22.733757 14871 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:33:22.734455 14871 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:33:22.735420 14871 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:33:22.735498 14871 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:33:22.735618 14871 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:33:22.735661 14871 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:33:22.747922 14871 rpc_server.cc:307] RPC server started. Bound to: 127.11.14.66:44881
I20260807 18:33:22.747987 15013 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.14.66:44881 every 8 connection(s)
I20260807 18:33:22.749034 14871 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127579900523-11321-0/minicluster-data/ts-1/data/info.pb
I20260807 18:33:22.756404 11321 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkz56XN/build/debug/bin/kudu as pid 14871
I20260807 18:33:22.756572 11321 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127579900523-11321-0/minicluster-data/ts-1/wal/instance
I20260807 18:33:22.756932 11321 external_mini_cluster.cc:1428] Setting key bb324790031598247d7a0e743ffe25f4
Aug 07 18:33:22 dist-test-slave-6k22 krb5kdc[14611](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127602, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.66@KRBTEST.COM for kudu/127.11.14.126@KRBTEST.COM
I20260807 18:33:22.766341 14704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:22.751238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.66:54131 (local address 127.11.14.126:44795)
0807 18:33:22.751439 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:22.751443 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:22.752451 (+ 1008us) server_negotiation.cc:407] Connection header received
0807 18:33:22.753941 (+ 1490us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:22.753944 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:22.754006 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:22.754116 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:33:22.755405 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:22.756000 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:22.757355 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:22.757639 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:22.757755 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:22.762123 (+ 4368us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:33:22.762128 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:33:22.762132 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:33:22.762172 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:33:22.763958 (+ 1786us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:22.764631 (+ 673us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:22.764635 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:22.764637 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:22.764736 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:22.765112 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:22.765115 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:22.765117 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:22.765333 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:33:22.765444 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:22.766028 (+ 584us) server_negotiation.cc:299] Negotiation successful
0807 18:33:22.766174 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260807 18:33:22.767275 15016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:22.751790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:44795 (local address 127.11.14.66:54131)
0807 18:33:22.752284 (+ 494us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:22.752332 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:22.753712 (+ 1380us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:22.754314 (+ 602us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:22.754322 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:22.754718 (+ 396us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:33:22.755236 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:22.755251 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:22.756134 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:22.756142 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:22.757217 (+ 1075us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:22.757228 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:22.757845 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:22.757850 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:22.758089 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:22.759652 (+ 1563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:33:22.759693 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:33:22.761912 (+ 2219us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:33:22.764104 (+ 2192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:33:22.764112 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:33:22.764136 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:33:22.764514 (+ 378us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:33:22.764838 (+ 324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:33:22.764841 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:33:22.764843 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:33:22.765001 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:33:22.765460 (+ 459us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:33:22.765468 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:33:22.765773 (+ 305us) client_negotiation.cc:769] Sending connection context
0807 18:33:22.765987 (+ 214us) client_negotiation.cc:240] Negotiation successful
0807 18:33:22.766357 (+ 370us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":282,"thread_start_us":121,"threads_started":1}
WARNING: no policy specified for kudu/127.11.14.67@KRBTEST.COM; defaulting to no policy
I20260807 18:33:22.768625 15014 heartbeater.cc:347] Connected to a master server at 127.11.14.126:44795
I20260807 18:33:22.768872 15014 heartbeater.cc:464] Registering TS with master...
I20260807 18:33:22.769371 15014 heartbeater.cc:511] Master 127.11.14.126:44795 requested a full tablet report, sending...
I20260807 18:33:22.770531 14659 ts_manager.cc:194] Registered new tserver with Master: 663ccd8f5faf48f9b14da93df82ad0e0 (127.11.14.66:44881)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.14.67@KRBTEST.COM" created.
I20260807 18:33:22.771320 14659 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.14.66@KRBTEST.COM'} at 127.11.14.66:54131
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.14.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.67.keytab.
Entry for principal kudu/127.11.14.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.67.keytab.
I20260807 18:33:22.792229 11321 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkz56XN/build/debug/bin/kudu
/tmp/dist-test-taskkz56XN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127579900523-11321-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127579900523-11321-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127579900523-11321-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127579900523-11321-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.14.67:0
--local_ip_for_outbound_sockets=127.11.14.67
--webserver_interface=127.11.14.67
--webserver_port=0
--tserver_master_addrs=127.11.14.126:44795
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.67.keytab
--principal=kudu/127.11.14.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.14.84:40503
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127579900523-11321-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127579900523-11321-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127579900523-11321-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:33:22.911334 15019 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:33:22.911715 15019 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:33:22.911801 15019 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:33:22.916036 15019 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:33:22.916141 15019 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:33:22.916253 15019 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.14.67
I20260807 18:33:22.921725 15019 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.14.84:40503
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127579900523-11321-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127579900523-11321-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.67.keytab
--principal=kudu/127.11.14.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.14.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127579900523-11321-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.14.67
--webserver_port=0
--tserver_master_addrs=127.11.14.126:44795
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15019
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.14.67
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127579900523-11321-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:33:22.922967 15019 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:33:22.924134 15019 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:33:22.935056 15024 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:33:22.935092 15019 server_base.cc:1034] running on GCE node
W20260807 18:33:22.935074 15025 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:33:22.935078 15027 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:33:22.935781 15019 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:33:22.936457 15019 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:33:22.937644 15019 hybrid_clock.cc:648] HybridClock initialized: now 1786127602937620 us; error 33 us; skew 500 ppm
Aug 07 18:33:22 dist-test-slave-6k22 krb5kdc[14611](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127602, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:33:22.940783 15019 init.cc:377] Logged in from keytab as kudu/127.11.14.67@KRBTEST.COM (short username kudu)
I20260807 18:33:22.941759 15019 webserver.cc:460] Webserver started at http://127.11.14.67:38459/ using document root <none> and password file <none>
I20260807 18:33:22.942384 15019 fs_manager.cc:359] Metadata directory not provided
I20260807 18:33:22.942483 15019 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:33:22.942759 15019 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:33:22.944975 15019 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127579900523-11321-0/minicluster-data/ts-2/data/instance:
uuid: "c9a7053f5d3c4e87bde435a5369b53d3"
format_stamp: "Formatted at 2026-08-07 18:33:22 on dist-test-slave-6k22"
server_key: "139ada1ed8e097d3876713a04a0ae8a9"
server_key_iv: "bd3fe3cdb678246b6b04da900f34766c"
server_key_version: "encryptionkey@0"
I20260807 18:33:22.945606 15019 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127579900523-11321-0/minicluster-data/ts-2/wal/instance:
uuid: "c9a7053f5d3c4e87bde435a5369b53d3"
format_stamp: "Formatted at 2026-08-07 18:33:22 on dist-test-slave-6k22"
server_key: "139ada1ed8e097d3876713a04a0ae8a9"
server_key_iv: "bd3fe3cdb678246b6b04da900f34766c"
server_key_version: "encryptionkey@0"
I20260807 18:33:22.949846 15019 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 18:33:22.952570 15034 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:33:22.953936 15019 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260807 18:33:22.954099 15019 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127579900523-11321-0/minicluster-data/ts-2/data,/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127579900523-11321-0/minicluster-data/ts-2/wal
uuid: "c9a7053f5d3c4e87bde435a5369b53d3"
format_stamp: "Formatted at 2026-08-07 18:33:22 on dist-test-slave-6k22"
server_key: "139ada1ed8e097d3876713a04a0ae8a9"
server_key_iv: "bd3fe3cdb678246b6b04da900f34766c"
server_key_version: "encryptionkey@0"
I20260807 18:33:22.954233 15019 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127579900523-11321-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127579900523-11321-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127579900523-11321-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:33:22.971762 15019 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:33:22.975073 15019 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:33:22.975314 15019 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:33:22.976038 15019 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:33:22.976953 15019 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:33:22.977028 15019 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:33:22.977105 15019 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:33:22.977159 15019 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:33:22.987385 15019 rpc_server.cc:307] RPC server started. Bound to: 127.11.14.67:45757
I20260807 18:33:22.987411 15158 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.14.67:45757 every 8 connection(s)
I20260807 18:33:22.988458 15019 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127579900523-11321-0/minicluster-data/ts-2/data/info.pb
I20260807 18:33:22.990156 11321 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkz56XN/build/debug/bin/kudu as pid 15019
I20260807 18:33:22.990310 11321 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786127579900523-11321-0/minicluster-data/ts-2/wal/instance
I20260807 18:33:22.990657 11321 external_mini_cluster.cc:1428] Setting key 39b0f034f2cabdf9ad4d398a6020c283
Aug 07 18:33:22 dist-test-slave-6k22 krb5kdc[14611](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127602, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.67@KRBTEST.COM for kudu/127.11.14.126@KRBTEST.COM
I20260807 18:33:23.002841 14704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:22.990488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.67:35837 (local address 127.11.14.126:44795)
0807 18:33:22.990667 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:22.990674 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:22.991695 (+ 1021us) server_negotiation.cc:407] Connection header received
0807 18:33:22.993290 (+ 1595us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:22.993299 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:22.993347 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:22.993416 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:33:22.994640 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:22.995170 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:22.996456 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:22.996722 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:22.996827 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:22.999376 (+ 2549us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:33:22.999379 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:33:22.999382 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:33:22.999408 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:33:23.000955 (+ 1547us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:23.001466 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:23.001469 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:23.001470 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:23.001546 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:23.001859 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:23.001861 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:23.001863 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:23.002037 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:33:23.002114 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:23.002610 (+ 496us) server_negotiation.cc:299] Negotiation successful
0807 18:33:23.002708 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260807 18:33:23.003973 15161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:22.990916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:44795 (local address 127.11.14.67:35837)
0807 18:33:22.991511 (+ 595us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:22.991593 (+ 82us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:22.993090 (+ 1497us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:22.993603 (+ 513us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:22.993610 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:22.993980 (+ 370us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:33:22.994507 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:22.994521 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:22.995306 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:22.995309 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:22.996326 (+ 1017us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:22.996336 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:22.996874 (+ 538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:22.996877 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:22.997024 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:22.997683 (+ 659us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:33:22.997701 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:33:22.999194 (+ 1493us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:33:23.001072 (+ 1878us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:33:23.001077 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:33:23.001090 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:33:23.001374 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:33:23.001628 (+ 254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:33:23.001630 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:33:23.001632 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:33:23.001769 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:33:23.002128 (+ 359us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:33:23.002133 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:33:23.002393 (+ 260us) client_negotiation.cc:769] Sending connection context
0807 18:33:23.002559 (+ 166us) client_negotiation.cc:240] Negotiation successful
0807 18:33:23.002838 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":343,"thread_start_us":146,"threads_started":1}
I20260807 18:33:23.005221 15159 heartbeater.cc:347] Connected to a master server at 127.11.14.126:44795
I20260807 18:33:23.005486 15159 heartbeater.cc:464] Registering TS with master...
I20260807 18:33:23.006023 15159 heartbeater.cc:511] Master 127.11.14.126:44795 requested a full tablet report, sending...
I20260807 18:33:23.007069 14659 ts_manager.cc:194] Registered new tserver with Master: c9a7053f5d3c4e87bde435a5369b53d3 (127.11.14.67:45757)
I20260807 18:33:23.007695 14659 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.14.67@KRBTEST.COM'} at 127.11.14.67:35837
I20260807 18:33:23.017298 11321 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 18:33:23 dist-test-slave-6k22 krb5kdc[14611](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127603, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 07 18:33:23 dist-test-slave-6k22 krb5kdc[14611](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127603, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.11.14.126@KRBTEST.COM
I20260807 18:33:23.045403 14704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:23.035881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54836 (local address 127.11.14.126:44795)
0807 18:33:23.036074 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:23.036083 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:23.036162 (+ 79us) server_negotiation.cc:407] Connection header received
0807 18:33:23.036281 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:23.036283 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:23.036331 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:23.036403 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:33:23.037159 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:23.037753 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:23.038757 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:23.039019 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:23.039123 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:23.041979 (+ 2856us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:33:23.041982 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:33:23.041985 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:33:23.042010 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:33:23.043377 (+ 1367us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:23.043908 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:23.043911 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:23.043912 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:23.043991 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:23.044290 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:23.044294 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:23.044295 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:23.044569 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:33:23.044751 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:23.045016 (+ 265us) server_negotiation.cc:299] Negotiation successful
0807 18:33:23.045139 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
W20260807 18:33:23.045913 14659 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:54836
I20260807 18:33:23.047384 11321 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkz56XN/build/debug/bin/kudu with pid 14728
I20260807 18:33:23.058128 11321 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkz56XN/build/debug/bin/kudu with pid 14871
I20260807 18:33:23.067503 11321 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkz56XN/build/debug/bin/kudu with pid 15019
I20260807 18:33:23.076057 11321 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkz56XN/build/debug/bin/kudu with pid 14627
2026-08-07T18:33:23Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (2323 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127579900523-11321-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:33:23 dist-test-slave-6k22 krb5kdc[15172](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:33:23 dist-test-slave-6k22 krb5kdc[15172](info): set up 2 sockets
Aug 07 18:33:23 dist-test-slave-6k22 krb5kdc[15172](info): commencing operation
krb5kdc: starting...
W20260807 18:33:24.251811 11321 mini_kdc.cc:121] Time spent starting KDC: real 1.150s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127579900523-11321-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127579900523-11321-0/krb5kdc/test-user.keytab.
Aug 07 18:33:24 dist-test-slave-6k22 krb5kdc[15172](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127604, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:33:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:33:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.14.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.14.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.14.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.126.keytab.
Entry for principal kudu/127.11.14.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.126.keytab.
I20260807 18:33:24.387025 11321 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkz56XN/build/debug/bin/kudu
/tmp/dist-test-taskkz56XN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127579900523-11321-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127579900523-11321-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127579900523-11321-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127579900523-11321-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.14.126:42349
--webserver_interface=127.11.14.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.126.keytab
--principal=kudu/127.11.14.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.14.84:44305
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.14.126:42349
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127579900523-11321-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127579900523-11321-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127579900523-11321-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:33:24.513582 15186 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:33:24.513952 15186 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:33:24.514094 15186 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:33:24.519259 15186 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:33:24.519357 15186 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:33:24.519397 15186 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:33:24.519434 15186 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:33:24.519476 15186 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:33:24.528930 15186 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.14.84:44305
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127579900523-11321-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127579900523-11321-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.14.126:42349
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.126.keytab
--principal=kudu/127.11.14.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.14.126:42349
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127579900523-11321-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.14.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15186
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127579900523-11321-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:33:24.530660 15186 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:33:24.532279 15186 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:33:24.543161 15195 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:33:24.543196 15192 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:33:24.543196 15193 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:33:24.543744 15186 server_base.cc:1034] running on GCE node
I20260807 18:33:24.544551 15186 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:33:24.545825 15186 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:33:24.547035 15186 hybrid_clock.cc:648] HybridClock initialized: now 1786127604547005 us; error 49 us; skew 500 ppm
Aug 07 18:33:24 dist-test-slave-6k22 krb5kdc[15172](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127604, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:33:24.550004 15186 init.cc:377] Logged in from keytab as kudu/127.11.14.126@KRBTEST.COM (short username kudu)
I20260807 18:33:24.550953 15186 webserver.cc:460] Webserver started at http://127.11.14.126:35469/ using document root <none> and password file <none>
I20260807 18:33:24.551622 15186 fs_manager.cc:359] Metadata directory not provided
I20260807 18:33:24.551723 15186 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:33:24.552006 15186 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:33:24.554242 15186 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127579900523-11321-0/minicluster-data/master-0/data/instance:
uuid: "b2149392ec22441db59ac916112b54a5"
format_stamp: "Formatted at 2026-08-07 18:33:24 on dist-test-slave-6k22"
server_key: "61fe4d0272e426219dab2e7ca65ea8bc"
server_key_iv: "964462db6445a6c777cad3cc01f02627"
server_key_version: "encryptionkey@0"
I20260807 18:33:24.554865 15186 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127579900523-11321-0/minicluster-data/master-0/wal/instance:
uuid: "b2149392ec22441db59ac916112b54a5"
format_stamp: "Formatted at 2026-08-07 18:33:24 on dist-test-slave-6k22"
server_key: "61fe4d0272e426219dab2e7ca65ea8bc"
server_key_iv: "964462db6445a6c777cad3cc01f02627"
server_key_version: "encryptionkey@0"
I20260807 18:33:24.558828 15186 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 18:33:24.561412 15203 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:33:24.562811 15186 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260807 18:33:24.562974 15186 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127579900523-11321-0/minicluster-data/master-0/data,/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127579900523-11321-0/minicluster-data/master-0/wal
uuid: "b2149392ec22441db59ac916112b54a5"
format_stamp: "Formatted at 2026-08-07 18:33:24 on dist-test-slave-6k22"
server_key: "61fe4d0272e426219dab2e7ca65ea8bc"
server_key_iv: "964462db6445a6c777cad3cc01f02627"
server_key_version: "encryptionkey@0"
I20260807 18:33:24.563097 15186 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127579900523-11321-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127579900523-11321-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127579900523-11321-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:33:24.587361 15186 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:33:24.590832 15186 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:33:24.591075 15186 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:33:24.598927 15186 rpc_server.cc:307] RPC server started. Bound to: 127.11.14.126:42349
I20260807 18:33:24.598956 15261 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.14.126:42349 every 8 connection(s)
I20260807 18:33:24.599968 15186 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127579900523-11321-0/minicluster-data/master-0/data/info.pb
I20260807 18:33:24.602790 15263 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:33:24.605518 11321 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkz56XN/build/debug/bin/kudu as pid 15186
I20260807 18:33:24.605682 11321 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127579900523-11321-0/minicluster-data/master-0/wal/instance
I20260807 18:33:24.605962 11321 external_mini_cluster.cc:1428] Setting key 4bd4672858ce0c0bb78104568c748296
I20260807 18:33:24.608959 15263 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b2149392ec22441db59ac916112b54a5: Bootstrap starting.
I20260807 18:33:24.611666 15263 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b2149392ec22441db59ac916112b54a5: Neither blocks nor log segments found. Creating new log.
I20260807 18:33:24.612582 15263 log.cc:824] T 00000000000000000000000000000000 P b2149392ec22441db59ac916112b54a5: Log is configured to *not* fsync() on all Append() calls
I20260807 18:33:24.615092 15263 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b2149392ec22441db59ac916112b54a5: No bootstrap required, opened a new log
Aug 07 18:33:24 dist-test-slave-6k22 krb5kdc[15172](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127604, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.14.126@KRBTEST.COM
I20260807 18:33:24.619606 15263 raft_consensus.cc:348] T 00000000000000000000000000000000 P b2149392ec22441db59ac916112b54a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2149392ec22441db59ac916112b54a5" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 42349 } }
I20260807 18:33:24.619930 15263 raft_consensus.cc:374] T 00000000000000000000000000000000 P b2149392ec22441db59ac916112b54a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:33:24.619994 15263 raft_consensus.cc:729] T 00000000000000000000000000000000 P b2149392ec22441db59ac916112b54a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2149392ec22441db59ac916112b54a5, State: Initialized, Role: FOLLOWER
I20260807 18:33:24.620492 15263 consensus_queue.cc:260] T 00000000000000000000000000000000 P b2149392ec22441db59ac916112b54a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2149392ec22441db59ac916112b54a5" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 42349 } }
I20260807 18:33:24.620669 15263 raft_consensus.cc:388] T 00000000000000000000000000000000 P b2149392ec22441db59ac916112b54a5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:33:24.620746 15263 raft_consensus.cc:482] T 00000000000000000000000000000000 P b2149392ec22441db59ac916112b54a5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:33:24.620851 15263 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b2149392ec22441db59ac916112b54a5 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:33:24.621796 15263 raft_consensus.cc:504] T 00000000000000000000000000000000 P b2149392ec22441db59ac916112b54a5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2149392ec22441db59ac916112b54a5" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 42349 } }
I20260807 18:33:24.622004 15266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:24.607415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40094 (local address 127.11.14.126:42349)
0807 18:33:24.607889 (+ 474us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:24.607907 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:24.607952 (+ 45us) server_negotiation.cc:407] Connection header received
0807 18:33:24.608565 (+ 613us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:24.608582 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:24.608935 (+ 353us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:24.609282 (+ 347us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:33:24.610185 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:24.611219 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:24.612438 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:24.613025 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:24.613206 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:24.616090 (+ 2884us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:33:24.616097 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:33:24.616115 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:33:24.616147 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:33:24.618831 (+ 2684us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:24.619290 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:24.619296 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:24.619302 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:24.619445 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:24.619794 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:24.619798 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:24.619800 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:24.620349 (+ 549us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:33:24.620552 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:24.620848 (+ 296us) server_negotiation.cc:299] Negotiation successful
0807 18:33:24.621192 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":321,"thread_start_us":185,"threads_started":1}
I20260807 18:33:24.622205 15263 leader_election.cc:302] T 00000000000000000000000000000000 P b2149392ec22441db59ac916112b54a5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b2149392ec22441db59ac916112b54a5; no voters:
I20260807 18:33:24.622584 15263 leader_election.cc:288] T 00000000000000000000000000000000 P b2149392ec22441db59ac916112b54a5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:33:24.622699 15268 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b2149392ec22441db59ac916112b54a5 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:33:24.622974 15268 raft_consensus.cc:686] T 00000000000000000000000000000000 P b2149392ec22441db59ac916112b54a5 [term 1 LEADER]: Becoming Leader. State: Replica: b2149392ec22441db59ac916112b54a5, State: Running, Role: LEADER
I20260807 18:33:24.623293 15268 consensus_queue.cc:237] T 00000000000000000000000000000000 P b2149392ec22441db59ac916112b54a5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2149392ec22441db59ac916112b54a5" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 42349 } }
I20260807 18:33:24.623956 15263 sys_catalog.cc:564] T 00000000000000000000000000000000 P b2149392ec22441db59ac916112b54a5 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:33:24.625088 15269 sys_catalog.cc:455] T 00000000000000000000000000000000 P b2149392ec22441db59ac916112b54a5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b2149392ec22441db59ac916112b54a5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2149392ec22441db59ac916112b54a5" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 42349 } } }
I20260807 18:33:24.625242 15269 sys_catalog.cc:458] T 00000000000000000000000000000000 P b2149392ec22441db59ac916112b54a5 [sys.catalog]: This master's current role is: LEADER
I20260807 18:33:24.625492 15271 sys_catalog.cc:455] T 00000000000000000000000000000000 P b2149392ec22441db59ac916112b54a5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b2149392ec22441db59ac916112b54a5. Latest consensus state: current_term: 1 leader_uuid: "b2149392ec22441db59ac916112b54a5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2149392ec22441db59ac916112b54a5" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 42349 } } }
I20260807 18:33:24.625634 15271 sys_catalog.cc:458] T 00000000000000000000000000000000 P b2149392ec22441db59ac916112b54a5 [sys.catalog]: This master's current role is: LEADER
I20260807 18:33:24.625670 15278 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:33:24.628891 15278 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:33:24.635038 15278 catalog_manager.cc:1347] Generated new cluster ID: 92c36fbf26634d4c855b17dcacccb334
I20260807 18:33:24.635131 15278 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:33:24.654031 15278 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:33:24.655087 15278 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:33:24.663887 15278 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b2149392ec22441db59ac916112b54a5: Generated new TSK 0
I20260807 18:33:24.664666 15278 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.14.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.14.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.14.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.65.keytab.
Entry for principal kudu/127.11.14.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.65.keytab.
I20260807 18:33:24.698289 11321 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkz56XN/build/debug/bin/kudu
/tmp/dist-test-taskkz56XN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127579900523-11321-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127579900523-11321-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127579900523-11321-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127579900523-11321-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.14.65:0
--local_ip_for_outbound_sockets=127.11.14.65
--webserver_interface=127.11.14.65
--webserver_port=0
--tserver_master_addrs=127.11.14.126:42349
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.65.keytab
--principal=kudu/127.11.14.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.14.84:44305
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127579900523-11321-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127579900523-11321-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127579900523-11321-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:33:24.820809 15292 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:33:24.821105 15292 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:33:24.821149 15292 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:33:24.825685 15292 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:33:24.825759 15292 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:33:24.825822 15292 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.14.65
I20260807 18:33:24.831422 15292 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.14.84:44305
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127579900523-11321-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127579900523-11321-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.65.keytab
--principal=kudu/127.11.14.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.14.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127579900523-11321-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.14.65
--webserver_port=0
--tserver_master_addrs=127.11.14.126:42349
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15292
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.14.65
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127579900523-11321-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:33:24.832641 15292 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:33:24.833735 15292 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:33:24.843811 15299 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:33:24.843864 15297 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:33:24.844105 15301 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:33:24.844156 15292 server_base.cc:1034] running on GCE node
I20260807 18:33:24.844632 15292 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:33:24.845393 15292 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:33:24.846701 15292 hybrid_clock.cc:648] HybridClock initialized: now 1786127604846674 us; error 44 us; skew 500 ppm
Aug 07 18:33:24 dist-test-slave-6k22 krb5kdc[15172](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127604, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:33:24.849967 15292 init.cc:377] Logged in from keytab as kudu/127.11.14.65@KRBTEST.COM (short username kudu)
I20260807 18:33:24.851544 15292 webserver.cc:460] Webserver started at http://127.11.14.65:38065/ using document root <none> and password file <none>
I20260807 18:33:24.852272 15292 fs_manager.cc:359] Metadata directory not provided
I20260807 18:33:24.852386 15292 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:33:24.852743 15292 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:33:24.855187 15292 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127579900523-11321-0/minicluster-data/ts-0/data/instance:
uuid: "d4c156442fe64691b6defee4580cf481"
format_stamp: "Formatted at 2026-08-07 18:33:24 on dist-test-slave-6k22"
server_key: "33cdcab89d0ed329d8cafe0ecd30379c"
server_key_iv: "8642845f954c9be0316b31a28db99115"
server_key_version: "encryptionkey@0"
I20260807 18:33:24.855899 15292 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127579900523-11321-0/minicluster-data/ts-0/wal/instance:
uuid: "d4c156442fe64691b6defee4580cf481"
format_stamp: "Formatted at 2026-08-07 18:33:24 on dist-test-slave-6k22"
server_key: "33cdcab89d0ed329d8cafe0ecd30379c"
server_key_iv: "8642845f954c9be0316b31a28db99115"
server_key_version: "encryptionkey@0"
I20260807 18:33:24.860387 15292 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260807 18:33:24.863334 15308 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:33:24.864780 15292 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 18:33:24.864954 15292 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127579900523-11321-0/minicluster-data/ts-0/data,/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127579900523-11321-0/minicluster-data/ts-0/wal
uuid: "d4c156442fe64691b6defee4580cf481"
format_stamp: "Formatted at 2026-08-07 18:33:24 on dist-test-slave-6k22"
server_key: "33cdcab89d0ed329d8cafe0ecd30379c"
server_key_iv: "8642845f954c9be0316b31a28db99115"
server_key_version: "encryptionkey@0"
I20260807 18:33:24.865103 15292 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127579900523-11321-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127579900523-11321-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127579900523-11321-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:33:24.884263 15292 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:33:24.887802 15292 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:33:24.888028 15292 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:33:24.888664 15292 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:33:24.889536 15292 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:33:24.889592 15292 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:33:24.889663 15292 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:33:24.889696 15292 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:33:24.899977 15292 rpc_server.cc:307] RPC server started. Bound to: 127.11.14.65:45023
I20260807 18:33:24.900058 15431 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.14.65:45023 every 8 connection(s)
I20260807 18:33:24.900974 15292 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127579900523-11321-0/minicluster-data/ts-0/data/info.pb
I20260807 18:33:24.902346 11321 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkz56XN/build/debug/bin/kudu as pid 15292
I20260807 18:33:24.902487 11321 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127579900523-11321-0/minicluster-data/ts-0/wal/instance
I20260807 18:33:24.902858 11321 external_mini_cluster.cc:1428] Setting key 19e7e092b724f903f2e0d424e71a1db6
Aug 07 18:33:24 dist-test-slave-6k22 krb5kdc[15172](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127604, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.65@KRBTEST.COM for kudu/127.11.14.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.11.14.66@KRBTEST.COM; defaulting to no policy
I20260807 18:33:24.917164 15266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:24.903081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.65:51767 (local address 127.11.14.126:42349)
0807 18:33:24.903281 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:24.903285 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:24.904466 (+ 1181us) server_negotiation.cc:407] Connection header received
0807 18:33:24.905938 (+ 1472us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:24.905941 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:24.905999 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:24.906107 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:33:24.907324 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:24.907919 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:24.909086 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:24.909332 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:24.909449 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:24.912640 (+ 3191us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:33:24.912643 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:33:24.912646 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:33:24.912671 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:33:24.914164 (+ 1493us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:24.915059 (+ 895us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:24.915063 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:24.915065 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:24.915185 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:24.915781 (+ 596us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:24.915784 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:24.915786 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:24.915997 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:33:24.916094 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:24.916706 (+ 612us) server_negotiation.cc:299] Negotiation successful
0807 18:33:24.916895 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.14.66@KRBTEST.COM" created.
I20260807 18:33:24.918009 15434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:24.903673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:42349 (local address 127.11.14.65:51767)
0807 18:33:24.904270 (+ 597us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:24.904330 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:24.905712 (+ 1382us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:24.906300 (+ 588us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:24.906306 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:24.906668 (+ 362us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:33:24.907166 (+ 498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:24.907180 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:24.908050 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:24.908054 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:24.908951 (+ 897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:24.908961 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:24.909496 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:24.909501 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:24.909692 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:24.910334 (+ 642us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:33:24.910352 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:33:24.912432 (+ 2080us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:33:24.914326 (+ 1894us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:33:24.914334 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:33:24.914355 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:33:24.914796 (+ 441us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:33:24.915314 (+ 518us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:33:24.915317 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:33:24.915318 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:33:24.915541 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:33:24.916112 (+ 571us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:33:24.916118 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:33:24.916401 (+ 283us) client_negotiation.cc:769] Sending connection context
0807 18:33:24.916578 (+ 177us) client_negotiation.cc:240] Negotiation successful
0807 18:33:24.916912 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":358,"thread_start_us":130,"threads_started":1}
I20260807 18:33:24.919651 15432 heartbeater.cc:347] Connected to a master server at 127.11.14.126:42349
I20260807 18:33:24.919960 15432 heartbeater.cc:464] Registering TS with master...
I20260807 18:33:24.920573 15432 heartbeater.cc:511] Master 127.11.14.126:42349 requested a full tablet report, sending...
I20260807 18:33:24.922315 15223 ts_manager.cc:194] Registered new tserver with Master: d4c156442fe64691b6defee4580cf481 (127.11.14.65:45023)
I20260807 18:33:24.923539 15223 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.14.65@KRBTEST.COM'} at 127.11.14.65:51767
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.14.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.66.keytab.
Entry for principal kudu/127.11.14.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.66.keytab.
I20260807 18:33:24.937036 11321 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkz56XN/build/debug/bin/kudu
/tmp/dist-test-taskkz56XN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127579900523-11321-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127579900523-11321-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127579900523-11321-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127579900523-11321-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.14.66:0
--local_ip_for_outbound_sockets=127.11.14.66
--webserver_interface=127.11.14.66
--webserver_port=0
--tserver_master_addrs=127.11.14.126:42349
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.66.keytab
--principal=kudu/127.11.14.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.14.84:44305
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127579900523-11321-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127579900523-11321-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127579900523-11321-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:33:25.056983 15437 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:33:25.057303 15437 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:33:25.057379 15437 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:33:25.061829 15437 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:33:25.061923 15437 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:33:25.062011 15437 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.14.66
I20260807 18:33:25.067755 15437 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.14.84:44305
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127579900523-11321-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127579900523-11321-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.66.keytab
--principal=kudu/127.11.14.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.14.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127579900523-11321-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.14.66
--webserver_port=0
--tserver_master_addrs=127.11.14.126:42349
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15437
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.14.66
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127579900523-11321-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:33:25.068926 15437 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:33:25.070075 15437 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:33:25.080363 15446 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:33:25.080374 15444 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:33:25.080432 15437 server_base.cc:1034] running on GCE node
W20260807 18:33:25.080379 15442 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:33:25.080967 15437 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:33:25.081620 15437 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:33:25.082798 15437 hybrid_clock.cc:648] HybridClock initialized: now 1786127605082770 us; error 40 us; skew 500 ppm
Aug 07 18:33:25 dist-test-slave-6k22 krb5kdc[15172](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127605, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:33:25.085772 15437 init.cc:377] Logged in from keytab as kudu/127.11.14.66@KRBTEST.COM (short username kudu)
I20260807 18:33:25.086732 15437 webserver.cc:460] Webserver started at http://127.11.14.66:34399/ using document root <none> and password file <none>
I20260807 18:33:25.087333 15437 fs_manager.cc:359] Metadata directory not provided
I20260807 18:33:25.087399 15437 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:33:25.087675 15437 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:33:25.089828 15437 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127579900523-11321-0/minicluster-data/ts-1/data/instance:
uuid: "761a382b1fdc4387b7ec473ccee099b4"
format_stamp: "Formatted at 2026-08-07 18:33:25 on dist-test-slave-6k22"
server_key: "ce75c544d92ece37e163e0c04c327d6a"
server_key_iv: "c67556e7e4e3659be29a426a3a93e5bd"
server_key_version: "encryptionkey@0"
I20260807 18:33:25.090461 15437 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127579900523-11321-0/minicluster-data/ts-1/wal/instance:
uuid: "761a382b1fdc4387b7ec473ccee099b4"
format_stamp: "Formatted at 2026-08-07 18:33:25 on dist-test-slave-6k22"
server_key: "ce75c544d92ece37e163e0c04c327d6a"
server_key_iv: "c67556e7e4e3659be29a426a3a93e5bd"
server_key_version: "encryptionkey@0"
I20260807 18:33:25.094704 15437 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260807 18:33:25.097499 15453 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:33:25.098896 15437 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 18:33:25.099028 15437 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127579900523-11321-0/minicluster-data/ts-1/data,/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127579900523-11321-0/minicluster-data/ts-1/wal
uuid: "761a382b1fdc4387b7ec473ccee099b4"
format_stamp: "Formatted at 2026-08-07 18:33:25 on dist-test-slave-6k22"
server_key: "ce75c544d92ece37e163e0c04c327d6a"
server_key_iv: "c67556e7e4e3659be29a426a3a93e5bd"
server_key_version: "encryptionkey@0"
I20260807 18:33:25.099145 15437 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127579900523-11321-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127579900523-11321-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127579900523-11321-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:33:25.109033 15437 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:33:25.112627 15437 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:33:25.112855 15437 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:33:25.113528 15437 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:33:25.114383 15437 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:33:25.114431 15437 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:33:25.114470 15437 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:33:25.114512 15437 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:33:25.124018 15437 rpc_server.cc:307] RPC server started. Bound to: 127.11.14.66:36139
I20260807 18:33:25.124071 15577 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.14.66:36139 every 8 connection(s)
I20260807 18:33:25.125208 15437 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127579900523-11321-0/minicluster-data/ts-1/data/info.pb
I20260807 18:33:25.135079 11321 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkz56XN/build/debug/bin/kudu as pid 15437
I20260807 18:33:25.135195 11321 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127579900523-11321-0/minicluster-data/ts-1/wal/instance
I20260807 18:33:25.135480 11321 external_mini_cluster.cc:1428] Setting key e45fef6ef304e41dcb49caea66185740
Aug 07 18:33:25 dist-test-slave-6k22 krb5kdc[15172](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127605, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.66@KRBTEST.COM for kudu/127.11.14.126@KRBTEST.COM
I20260807 18:33:25.140599 15266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:25.127157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.66:46101 (local address 127.11.14.126:42349)
0807 18:33:25.127307 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:25.127311 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:25.128348 (+ 1037us) server_negotiation.cc:407] Connection header received
0807 18:33:25.129854 (+ 1506us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:25.129858 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:25.129945 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:25.130056 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:33:25.131307 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:25.131907 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:25.133079 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:25.133309 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:25.133421 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:25.136311 (+ 2890us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:33:25.136314 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:33:25.136317 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:33:25.136348 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:33:25.137880 (+ 1532us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:25.138559 (+ 679us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:25.138562 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:25.138563 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:25.138643 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:25.139042 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:25.139045 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:25.139046 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:25.139326 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:33:25.139437 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:25.140223 (+ 786us) server_negotiation.cc:299] Negotiation successful
0807 18:33:25.140375 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260807 18:33:25.141111 15580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:25.127621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:42349 (local address 127.11.14.66:46101)
0807 18:33:25.128166 (+ 545us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:25.128217 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:25.129608 (+ 1391us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:25.130247 (+ 639us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:25.130255 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:25.130640 (+ 385us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:33:25.131145 (+ 505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:25.131158 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:25.132045 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:25.132048 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:25.132939 (+ 891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:25.132948 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:25.133471 (+ 523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:25.133476 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:25.133642 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:25.134313 (+ 671us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:33:25.134330 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:33:25.136090 (+ 1760us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:33:25.138009 (+ 1919us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:33:25.138019 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:33:25.138039 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:33:25.138452 (+ 413us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:33:25.138734 (+ 282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:33:25.138740 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:33:25.138745 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:33:25.138947 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:33:25.139444 (+ 497us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:33:25.139450 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:33:25.139772 (+ 322us) client_negotiation.cc:769] Sending connection context
0807 18:33:25.139939 (+ 167us) client_negotiation.cc:240] Negotiation successful
0807 18:33:25.140217 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":317,"thread_start_us":119,"threads_started":1}
I20260807 18:33:25.142448 15578 heartbeater.cc:347] Connected to a master server at 127.11.14.126:42349
I20260807 18:33:25.142776 15578 heartbeater.cc:464] Registering TS with master...
I20260807 18:33:25.143313 15578 heartbeater.cc:511] Master 127.11.14.126:42349 requested a full tablet report, sending...
I20260807 18:33:25.144534 15223 ts_manager.cc:194] Registered new tserver with Master: 761a382b1fdc4387b7ec473ccee099b4 (127.11.14.66:36139)
I20260807 18:33:25.145390 15223 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.14.66@KRBTEST.COM'} at 127.11.14.66:46101
WARNING: no policy specified for kudu/127.11.14.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.14.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.14.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.67.keytab.
Entry for principal kudu/127.11.14.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.67.keytab.
I20260807 18:33:25.166818 11321 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkz56XN/build/debug/bin/kudu
/tmp/dist-test-taskkz56XN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127579900523-11321-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127579900523-11321-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127579900523-11321-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127579900523-11321-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.14.67:0
--local_ip_for_outbound_sockets=127.11.14.67
--webserver_interface=127.11.14.67
--webserver_port=0
--tserver_master_addrs=127.11.14.126:42349
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.67.keytab
--principal=kudu/127.11.14.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.14.84:44305
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127579900523-11321-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127579900523-11321-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127579900523-11321-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:33:25.285516 15586 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:33:25.285825 15586 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:33:25.285876 15586 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:33:25.290202 15586 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:33:25.290273 15586 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:33:25.290337 15586 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.14.67
I20260807 18:33:25.295885 15586 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.14.84:44305
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127579900523-11321-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127579900523-11321-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.67.keytab
--principal=kudu/127.11.14.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.14.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127579900523-11321-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.14.67
--webserver_port=0
--tserver_master_addrs=127.11.14.126:42349
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15586
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.14.67
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127579900523-11321-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:33:25.297197 15586 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:33:25.298310 15586 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:33:25.308722 15593 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:33:25.308717 15595 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:33:25.308732 15592 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:33:25.308939 15586 server_base.cc:1034] running on GCE node
I20260807 18:33:25.309469 15586 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:33:25.310088 15586 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:33:25.311302 15586 hybrid_clock.cc:648] HybridClock initialized: now 1786127605311275 us; error 37 us; skew 500 ppm
Aug 07 18:33:25 dist-test-slave-6k22 krb5kdc[15172](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127605, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:33:25.314446 15586 init.cc:377] Logged in from keytab as kudu/127.11.14.67@KRBTEST.COM (short username kudu)
I20260807 18:33:25.315398 15586 webserver.cc:460] Webserver started at http://127.11.14.67:33827/ using document root <none> and password file <none>
I20260807 18:33:25.316046 15586 fs_manager.cc:359] Metadata directory not provided
I20260807 18:33:25.316110 15586 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:33:25.316345 15586 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:33:25.318545 15586 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127579900523-11321-0/minicluster-data/ts-2/data/instance:
uuid: "db25f49b2c074569a3d3b2bbdd7b98ab"
format_stamp: "Formatted at 2026-08-07 18:33:25 on dist-test-slave-6k22"
server_key: "dd40c240da2147d7c2537fcd62f07792"
server_key_iv: "f7d076a4fef6579763998bfcc8747747"
server_key_version: "encryptionkey@0"
I20260807 18:33:25.319142 15586 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127579900523-11321-0/minicluster-data/ts-2/wal/instance:
uuid: "db25f49b2c074569a3d3b2bbdd7b98ab"
format_stamp: "Formatted at 2026-08-07 18:33:25 on dist-test-slave-6k22"
server_key: "dd40c240da2147d7c2537fcd62f07792"
server_key_iv: "f7d076a4fef6579763998bfcc8747747"
server_key_version: "encryptionkey@0"
I20260807 18:33:25.323127 15586 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 18:33:25.326066 15603 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:33:25.327396 15586 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 18:33:25.327540 15586 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127579900523-11321-0/minicluster-data/ts-2/data,/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127579900523-11321-0/minicluster-data/ts-2/wal
uuid: "db25f49b2c074569a3d3b2bbdd7b98ab"
format_stamp: "Formatted at 2026-08-07 18:33:25 on dist-test-slave-6k22"
server_key: "dd40c240da2147d7c2537fcd62f07792"
server_key_iv: "f7d076a4fef6579763998bfcc8747747"
server_key_version: "encryptionkey@0"
I20260807 18:33:25.327718 15586 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127579900523-11321-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127579900523-11321-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127579900523-11321-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:33:25.339274 15586 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:33:25.342574 15586 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:33:25.342799 15586 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:33:25.343429 15586 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:33:25.344336 15586 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:33:25.344411 15586 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:33:25.344467 15586 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:33:25.344488 15586 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:33:25.353158 15586 rpc_server.cc:307] RPC server started. Bound to: 127.11.14.67:46601
I20260807 18:33:25.353204 15724 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.14.67:46601 every 8 connection(s)
I20260807 18:33:25.354106 15586 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127579900523-11321-0/minicluster-data/ts-2/data/info.pb
I20260807 18:33:25.354988 11321 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkz56XN/build/debug/bin/kudu as pid 15586
I20260807 18:33:25.355136 11321 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786127579900523-11321-0/minicluster-data/ts-2/wal/instance
I20260807 18:33:25.355427 11321 external_mini_cluster.cc:1428] Setting key f76ae86af00b6dfde87955e748da5db8
Aug 07 18:33:25 dist-test-slave-6k22 krb5kdc[15172](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127605, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.67@KRBTEST.COM for kudu/127.11.14.126@KRBTEST.COM
I20260807 18:33:25.370007 15266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:25.356635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.67:37603 (local address 127.11.14.126:42349)
0807 18:33:25.356771 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:25.356775 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:25.357862 (+ 1087us) server_negotiation.cc:407] Connection header received
0807 18:33:25.359383 (+ 1521us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:25.359387 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:25.359442 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:25.359522 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:33:25.360752 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:25.361300 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:25.362495 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:25.362750 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:25.362871 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:25.365561 (+ 2690us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:33:25.365565 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:33:25.365568 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:33:25.365602 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:33:25.367231 (+ 1629us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:25.367859 (+ 628us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:25.367862 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:25.367863 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:25.367977 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:25.368503 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:25.368507 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:25.368509 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:25.368816 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:33:25.368953 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:25.369667 (+ 714us) server_negotiation.cc:299] Negotiation successful
0807 18:33:25.369847 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260807 18:33:25.370985 15727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:25.357145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:42349 (local address 127.11.14.67:37603)
0807 18:33:25.357693 (+ 548us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:25.357762 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:25.359171 (+ 1409us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:25.359724 (+ 553us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:25.359732 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:25.360112 (+ 380us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:33:25.360605 (+ 493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:25.360618 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:25.361437 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:25.361440 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:25.362362 (+ 922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:25.362372 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:25.362916 (+ 544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:25.362921 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:25.363096 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:25.363769 (+ 673us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:33:25.363793 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:33:25.365384 (+ 1591us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:33:25.367362 (+ 1978us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:33:25.367368 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:33:25.367384 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:33:25.367732 (+ 348us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:33:25.368071 (+ 339us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:33:25.368075 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:33:25.368077 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:33:25.368286 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:33:25.368964 (+ 678us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:33:25.368970 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:33:25.369335 (+ 365us) client_negotiation.cc:769] Sending connection context
0807 18:33:25.369551 (+ 216us) client_negotiation.cc:240] Negotiation successful
0807 18:33:25.370057 (+ 506us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":327,"thread_start_us":157,"threads_started":1}
I20260807 18:33:25.372216 15725 heartbeater.cc:347] Connected to a master server at 127.11.14.126:42349
I20260807 18:33:25.372496 15725 heartbeater.cc:464] Registering TS with master...
I20260807 18:33:25.373059 15725 heartbeater.cc:511] Master 127.11.14.126:42349 requested a full tablet report, sending...
I20260807 18:33:25.374109 15223 ts_manager.cc:194] Registered new tserver with Master: db25f49b2c074569a3d3b2bbdd7b98ab (127.11.14.67:46601)
I20260807 18:33:25.374707 15223 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.14.67@KRBTEST.COM'} at 127.11.14.67:37603
I20260807 18:33:25.382455 11321 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 18:33:25 dist-test-slave-6k22 krb5kdc[15172](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127605, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 07 18:33:25 dist-test-slave-6k22 krb5kdc[15172](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127605, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.14.65@KRBTEST.COM
I20260807 18:33:25.411195 15733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:25.401234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40474 (local address 127.11.14.65:45023)
0807 18:33:25.401611 (+ 377us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:25.401646 (+ 35us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:25.401680 (+ 34us) server_negotiation.cc:407] Connection header received
0807 18:33:25.401745 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:25.401750 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:25.401956 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:25.402093 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:33:25.402913 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:25.403827 (+ 914us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:25.404668 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:25.405054 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:25.405155 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:25.407779 (+ 2624us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:33:25.407789 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:33:25.407803 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:33:25.407828 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:33:25.409336 (+ 1508us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:25.409763 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:25.409769 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:25.409770 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:25.409852 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:25.410132 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:25.410135 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:25.410136 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:25.410424 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:33:25.410623 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:25.410757 (+ 134us) server_negotiation.cc:299] Negotiation successful
0807 18:33:25.410946 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":226,"thread_start_us":107,"threads_started":1}
Aug 07 18:33:25 dist-test-slave-6k22 krb5kdc[15172](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127605, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.14.126@KRBTEST.COM
I20260807 18:33:25.422940 15266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:25.414443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40098 (local address 127.11.14.126:42349)
0807 18:33:25.414586 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:25.414590 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:25.414666 (+ 76us) server_negotiation.cc:407] Connection header received
0807 18:33:25.414765 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:25.414767 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:25.414811 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:25.414885 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:33:25.415467 (+ 582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:25.416042 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:25.416778 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:25.416980 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:25.417057 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:25.419730 (+ 2673us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:33:25.419734 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:33:25.419747 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:33:25.419775 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:33:25.421249 (+ 1474us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:25.421699 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:25.421702 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:25.421703 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:25.421787 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:25.422029 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:25.422032 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:25.422033 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:25.422307 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:33:25.422425 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:25.422682 (+ 257us) server_negotiation.cc:299] Negotiation successful
0807 18:33:25.422796 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
W20260807 18:33:25.423434 15223 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40098
I20260807 18:33:25.424309 11321 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkz56XN/build/debug/bin/kudu with pid 15292
I20260807 18:33:25.434967 11321 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkz56XN/build/debug/bin/kudu with pid 15437
I20260807 18:33:25.446790 11321 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkz56XN/build/debug/bin/kudu with pid 15586
I20260807 18:33:25.455727 11321 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkz56XN/build/debug/bin/kudu with pid 15186
2026-08-07T18:33:25Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (2379 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127579900523-11321-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:33:25 dist-test-slave-6k22 krb5kdc[15739](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:33:25 dist-test-slave-6k22 krb5kdc[15739](info): set up 2 sockets
Aug 07 18:33:25 dist-test-slave-6k22 krb5kdc[15739](info): commencing operation
krb5kdc: starting...
W20260807 18:33:26.579990 11321 mini_kdc.cc:121] Time spent starting KDC: real 1.099s user 0.006s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127579900523-11321-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127579900523-11321-0/krb5kdc/test-user.keytab.
Aug 07 18:33:26 dist-test-slave-6k22 krb5kdc[15739](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127606, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:33:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:33:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.14.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.14.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.14.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.126.keytab.
Entry for principal kudu/127.11.14.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.126.keytab.
I20260807 18:33:26.714643 11321 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkz56XN/build/debug/bin/kudu
/tmp/dist-test-taskkz56XN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127579900523-11321-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127579900523-11321-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127579900523-11321-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127579900523-11321-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.14.126:38005
--webserver_interface=127.11.14.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.126.keytab
--principal=kudu/127.11.14.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.14.84:40365
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.14.126:38005
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127579900523-11321-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127579900523-11321-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127579900523-11321-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:33:26.835311 15753 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:33:26.835668 15753 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:33:26.835754 15753 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:33:26.839923 15753 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:33:26.839990 15753 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:33:26.840010 15753 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:33:26.840029 15753 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:33:26.840044 15753 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:33:26.845463 15753 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.14.84:40365
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127579900523-11321-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127579900523-11321-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.14.126:38005
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.126.keytab
--principal=kudu/127.11.14.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.14.126:38005
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127579900523-11321-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.14.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15753
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127579900523-11321-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:33:26.846618 15753 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:33:26.847731 15753 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 18:33:26.856554 15753 server_base.cc:1034] running on GCE node
W20260807 18:33:26.856514 15761 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:33:26.856513 15759 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:33:26.856514 15758 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:33:26.857378 15753 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:33:26.858618 15753 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:33:26.859833 15753 hybrid_clock.cc:648] HybridClock initialized: now 1786127606859818 us; error 26 us; skew 500 ppm
Aug 07 18:33:26 dist-test-slave-6k22 krb5kdc[15739](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127606, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:33:26.862871 15753 init.cc:377] Logged in from keytab as kudu/127.11.14.126@KRBTEST.COM (short username kudu)
I20260807 18:33:26.863830 15753 fs_manager.cc:359] Metadata directory not provided
I20260807 18:33:26.863927 15753 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:33:26.864151 15753 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:33:26.866273 15753 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127579900523-11321-0/minicluster-data/master-0/data/instance:
uuid: "b53a559e96c74f7aa8e56f91a49ab0b8"
format_stamp: "Formatted at 2026-08-07 18:33:26 on dist-test-slave-6k22"
server_key: "7970191f1c5b33de070719cf7cacb1b4"
server_key_iv: "21f13d4e4007dc24a48e8faf6f814ff0"
server_key_version: "encryptionkey@0"
I20260807 18:33:26.866851 15753 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127579900523-11321-0/minicluster-data/master-0/wal/instance:
uuid: "b53a559e96c74f7aa8e56f91a49ab0b8"
format_stamp: "Formatted at 2026-08-07 18:33:26 on dist-test-slave-6k22"
server_key: "7970191f1c5b33de070719cf7cacb1b4"
server_key_iv: "21f13d4e4007dc24a48e8faf6f814ff0"
server_key_version: "encryptionkey@0"
I20260807 18:33:26.870708 15753 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 18:33:26.873339 15768 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:33:26.874679 15753 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 18:33:26.874825 15753 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127579900523-11321-0/minicluster-data/master-0/data,/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127579900523-11321-0/minicluster-data/master-0/wal
uuid: "b53a559e96c74f7aa8e56f91a49ab0b8"
format_stamp: "Formatted at 2026-08-07 18:33:26 on dist-test-slave-6k22"
server_key: "7970191f1c5b33de070719cf7cacb1b4"
server_key_iv: "21f13d4e4007dc24a48e8faf6f814ff0"
server_key_version: "encryptionkey@0"
I20260807 18:33:26.874939 15753 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127579900523-11321-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127579900523-11321-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127579900523-11321-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:33:26.892235 15753 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:33:26.895457 15753 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:33:26.895709 15753 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:33:26.903525 15753 rpc_server.cc:307] RPC server started. Bound to: 127.11.14.126:38005
I20260807 18:33:26.903970 15824 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.14.126:38005 every 8 connection(s)
I20260807 18:33:26.904282 15753 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127579900523-11321-0/minicluster-data/master-0/data/info.pb
I20260807 18:33:26.907397 15825 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:33:26.912612 11321 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkz56XN/build/debug/bin/kudu as pid 15753
I20260807 18:33:26.912775 11321 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127579900523-11321-0/minicluster-data/master-0/wal/instance
I20260807 18:33:26.913066 11321 external_mini_cluster.cc:1428] Setting key 535a3335367119f42d2d33e556869b9e
I20260807 18:33:26.914309 15825 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b53a559e96c74f7aa8e56f91a49ab0b8: Bootstrap starting.
I20260807 18:33:26.917150 15825 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b53a559e96c74f7aa8e56f91a49ab0b8: Neither blocks nor log segments found. Creating new log.
I20260807 18:33:26.918143 15825 log.cc:824] T 00000000000000000000000000000000 P b53a559e96c74f7aa8e56f91a49ab0b8: Log is configured to *not* fsync() on all Append() calls
I20260807 18:33:26.920802 15825 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b53a559e96c74f7aa8e56f91a49ab0b8: No bootstrap required, opened a new log
Aug 07 18:33:26 dist-test-slave-6k22 krb5kdc[15739](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127606, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.14.126@KRBTEST.COM
I20260807 18:33:26.925827 15825 raft_consensus.cc:348] T 00000000000000000000000000000000 P b53a559e96c74f7aa8e56f91a49ab0b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b53a559e96c74f7aa8e56f91a49ab0b8" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 38005 } }
I20260807 18:33:26.926092 15825 raft_consensus.cc:374] T 00000000000000000000000000000000 P b53a559e96c74f7aa8e56f91a49ab0b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:33:26.926229 15825 raft_consensus.cc:729] T 00000000000000000000000000000000 P b53a559e96c74f7aa8e56f91a49ab0b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b53a559e96c74f7aa8e56f91a49ab0b8, State: Initialized, Role: FOLLOWER
I20260807 18:33:26.926812 15825 consensus_queue.cc:260] T 00000000000000000000000000000000 P b53a559e96c74f7aa8e56f91a49ab0b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b53a559e96c74f7aa8e56f91a49ab0b8" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 38005 } }
I20260807 18:33:26.927062 15825 raft_consensus.cc:388] T 00000000000000000000000000000000 P b53a559e96c74f7aa8e56f91a49ab0b8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:33:26.927188 15825 raft_consensus.cc:482] T 00000000000000000000000000000000 P b53a559e96c74f7aa8e56f91a49ab0b8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:33:26.927356 15825 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b53a559e96c74f7aa8e56f91a49ab0b8 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:33:26.928845 15825 raft_consensus.cc:504] T 00000000000000000000000000000000 P b53a559e96c74f7aa8e56f91a49ab0b8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b53a559e96c74f7aa8e56f91a49ab0b8" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 38005 } }
I20260807 18:33:26.929260 15825 leader_election.cc:302] T 00000000000000000000000000000000 P b53a559e96c74f7aa8e56f91a49ab0b8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b53a559e96c74f7aa8e56f91a49ab0b8; no voters:
I20260807 18:33:26.929636 15825 leader_election.cc:288] T 00000000000000000000000000000000 P b53a559e96c74f7aa8e56f91a49ab0b8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:33:26.929744 15830 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b53a559e96c74f7aa8e56f91a49ab0b8 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:33:26.929750 15828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:26.914618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33058 (local address 127.11.14.126:38005)
0807 18:33:26.915086 (+ 468us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:26.915109 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:26.915153 (+ 44us) server_negotiation.cc:407] Connection header received
0807 18:33:26.915871 (+ 718us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:26.915891 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:26.916237 (+ 346us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:26.916521 (+ 284us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:33:26.917712 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:26.918936 (+ 1224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:26.920331 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:26.920791 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:26.920952 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:26.924192 (+ 3240us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:33:26.924198 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:33:26.924213 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:33:26.924244 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:33:26.926093 (+ 1849us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:26.926681 (+ 588us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:26.926688 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:26.926695 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:26.926871 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:26.927257 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:26.927260 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:26.927261 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:26.927852 (+ 591us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:33:26.928023 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:26.928406 (+ 383us) server_negotiation.cc:299] Negotiation successful
0807 18:33:26.928742 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":278,"thread_start_us":144,"threads_started":1}
I20260807 18:33:26.930032 15830 raft_consensus.cc:686] T 00000000000000000000000000000000 P b53a559e96c74f7aa8e56f91a49ab0b8 [term 1 LEADER]: Becoming Leader. State: Replica: b53a559e96c74f7aa8e56f91a49ab0b8, State: Running, Role: LEADER
I20260807 18:33:26.930372 15830 consensus_queue.cc:237] T 00000000000000000000000000000000 P b53a559e96c74f7aa8e56f91a49ab0b8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b53a559e96c74f7aa8e56f91a49ab0b8" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 38005 } }
I20260807 18:33:26.931244 15825 sys_catalog.cc:564] T 00000000000000000000000000000000 P b53a559e96c74f7aa8e56f91a49ab0b8 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:33:26.931684 15832 sys_catalog.cc:455] T 00000000000000000000000000000000 P b53a559e96c74f7aa8e56f91a49ab0b8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b53a559e96c74f7aa8e56f91a49ab0b8. Latest consensus state: current_term: 1 leader_uuid: "b53a559e96c74f7aa8e56f91a49ab0b8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b53a559e96c74f7aa8e56f91a49ab0b8" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 38005 } } }
I20260807 18:33:26.932479 15832 sys_catalog.cc:458] T 00000000000000000000000000000000 P b53a559e96c74f7aa8e56f91a49ab0b8 [sys.catalog]: This master's current role is: LEADER
I20260807 18:33:26.932358 15831 sys_catalog.cc:455] T 00000000000000000000000000000000 P b53a559e96c74f7aa8e56f91a49ab0b8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b53a559e96c74f7aa8e56f91a49ab0b8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b53a559e96c74f7aa8e56f91a49ab0b8" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 38005 } } }
I20260807 18:33:26.932837 15831 sys_catalog.cc:458] T 00000000000000000000000000000000 P b53a559e96c74f7aa8e56f91a49ab0b8 [sys.catalog]: This master's current role is: LEADER
I20260807 18:33:26.933677 15839 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:33:26.936643 15839 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:33:26.942137 15839 catalog_manager.cc:1347] Generated new cluster ID: 27517a31697e4edfab2b68695da6cd70
I20260807 18:33:26.942212 15839 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:33:26.963359 15839 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:33:26.964459 15839 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:33:26.972975 15839 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b53a559e96c74f7aa8e56f91a49ab0b8: Generated new TSK 0
I20260807 18:33:26.973742 15839 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.14.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.14.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.14.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.65.keytab.
Entry for principal kudu/127.11.14.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.65.keytab.
I20260807 18:33:27.015110 11321 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkz56XN/build/debug/bin/kudu
/tmp/dist-test-taskkz56XN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127579900523-11321-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127579900523-11321-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127579900523-11321-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127579900523-11321-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.14.65:0
--local_ip_for_outbound_sockets=127.11.14.65
--webserver_interface=127.11.14.65
--webserver_port=0
--tserver_master_addrs=127.11.14.126:38005
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.65.keytab
--principal=kudu/127.11.14.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.14.84:40365
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127579900523-11321-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127579900523-11321-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127579900523-11321-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:33:27.139544 15852 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:33:27.139869 15852 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:33:27.139918 15852 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:33:27.143926 15852 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:33:27.143996 15852 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:33:27.144059 15852 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.14.65
I20260807 18:33:27.149479 15852 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.14.84:40365
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127579900523-11321-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127579900523-11321-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.65.keytab
--principal=kudu/127.11.14.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.14.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127579900523-11321-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.14.65
--webserver_port=0
--tserver_master_addrs=127.11.14.126:38005
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15852
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.14.65
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127579900523-11321-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:33:27.150564 15852 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:33:27.151747 15852 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:33:27.161839 15858 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:33:27.161936 15852 server_base.cc:1034] running on GCE node
W20260807 18:33:27.161883 15861 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:33:27.161842 15859 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:33:27.162560 15852 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:33:27.163306 15852 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:33:27.164531 15852 hybrid_clock.cc:648] HybridClock initialized: now 1786127607164500 us; error 38 us; skew 500 ppm
Aug 07 18:33:27 dist-test-slave-6k22 krb5kdc[15739](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127607, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:33:27.173900 15852 init.cc:377] Logged in from keytab as kudu/127.11.14.65@KRBTEST.COM (short username kudu)
I20260807 18:33:27.174718 15852 fs_manager.cc:359] Metadata directory not provided
I20260807 18:33:27.174789 15852 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:33:27.174973 15852 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:33:27.177453 15852 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127579900523-11321-0/minicluster-data/ts-0/data/instance:
uuid: "215f5426ceae453d982d600d5e9ee2ab"
format_stamp: "Formatted at 2026-08-07 18:33:27 on dist-test-slave-6k22"
server_key: "b2d2974385fb21ddcab02388bda651f9"
server_key_iv: "4625344e513eadec8ea296d8744fc9af"
server_key_version: "encryptionkey@0"
I20260807 18:33:27.178071 15852 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127579900523-11321-0/minicluster-data/ts-0/wal/instance:
uuid: "215f5426ceae453d982d600d5e9ee2ab"
format_stamp: "Formatted at 2026-08-07 18:33:27 on dist-test-slave-6k22"
server_key: "b2d2974385fb21ddcab02388bda651f9"
server_key_iv: "4625344e513eadec8ea296d8744fc9af"
server_key_version: "encryptionkey@0"
I20260807 18:33:27.182449 15852 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 18:33:27.185254 15868 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:33:27.186636 15852 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 18:33:27.186849 15852 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127579900523-11321-0/minicluster-data/ts-0/data,/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127579900523-11321-0/minicluster-data/ts-0/wal
uuid: "215f5426ceae453d982d600d5e9ee2ab"
format_stamp: "Formatted at 2026-08-07 18:33:27 on dist-test-slave-6k22"
server_key: "b2d2974385fb21ddcab02388bda651f9"
server_key_iv: "4625344e513eadec8ea296d8744fc9af"
server_key_version: "encryptionkey@0"
I20260807 18:33:27.186975 15852 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127579900523-11321-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127579900523-11321-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127579900523-11321-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:33:27.198519 15852 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:33:27.202433 15852 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:33:27.202672 15852 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:33:27.203326 15852 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:33:27.204350 15852 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:33:27.204427 15852 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:33:27.204509 15852 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:33:27.204563 15852 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:33:27.214854 15852 rpc_server.cc:307] RPC server started. Bound to: 127.11.14.65:39823
I20260807 18:33:27.214877 15987 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.14.65:39823 every 8 connection(s)
I20260807 18:33:27.215525 15852 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127579900523-11321-0/minicluster-data/ts-0/data/info.pb
I20260807 18:33:27.224237 11321 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkz56XN/build/debug/bin/kudu as pid 15852
I20260807 18:33:27.224391 11321 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127579900523-11321-0/minicluster-data/ts-0/wal/instance
I20260807 18:33:27.224700 11321 external_mini_cluster.cc:1428] Setting key 98f8bd69afd10bf7e09a09a2978c7bd3
Aug 07 18:33:27 dist-test-slave-6k22 krb5kdc[15739](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127607, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.65@KRBTEST.COM for kudu/127.11.14.126@KRBTEST.COM
I20260807 18:33:27.230170 15828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:27.217395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.65:52463 (local address 127.11.14.126:38005)
0807 18:33:27.217580 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:27.217584 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:27.218487 (+ 903us) server_negotiation.cc:407] Connection header received
0807 18:33:27.219884 (+ 1397us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:27.219887 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:27.219936 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:27.220015 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:33:27.221265 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:27.221784 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:27.222917 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:27.223157 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:27.223240 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:27.226242 (+ 3002us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:33:27.226247 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:33:27.226249 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:33:27.226279 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:33:27.227938 (+ 1659us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:27.228476 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:27.228480 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:27.228482 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:27.228563 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:27.228921 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:27.228926 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:27.228927 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:27.229110 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:33:27.229227 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:27.229849 (+ 622us) server_negotiation.cc:299] Negotiation successful
0807 18:33:27.230010 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260807 18:33:27.231280 15990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:27.217788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:38005 (local address 127.11.14.65:52463)
0807 18:33:27.218298 (+ 510us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:27.218359 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:27.219665 (+ 1306us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:27.220189 (+ 524us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:27.220196 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:27.220559 (+ 363us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:33:27.221110 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:27.221123 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:27.221900 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:27.221903 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:27.222791 (+ 888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:27.222800 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:27.223299 (+ 499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:27.223302 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:27.223447 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:27.224102 (+ 655us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:33:27.224126 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:33:27.226046 (+ 1920us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:33:27.228065 (+ 2019us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:33:27.228072 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:33:27.228090 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:33:27.228367 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:33:27.228666 (+ 299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:33:27.228669 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:33:27.228671 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:33:27.228821 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:33:27.229227 (+ 406us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:33:27.229235 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:33:27.229491 (+ 256us) client_negotiation.cc:769] Sending connection context
0807 18:33:27.229712 (+ 221us) client_negotiation.cc:240] Negotiation successful
0807 18:33:27.230095 (+ 383us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":291,"thread_start_us":135,"threads_started":1}
I20260807 18:33:27.232771 15988 heartbeater.cc:347] Connected to a master server at 127.11.14.126:38005
I20260807 18:33:27.233038 15988 heartbeater.cc:464] Registering TS with master...
I20260807 18:33:27.233569 15988 heartbeater.cc:511] Master 127.11.14.126:38005 requested a full tablet report, sending...
I20260807 18:33:27.235219 15786 ts_manager.cc:194] Registered new tserver with Master: 215f5426ceae453d982d600d5e9ee2ab (127.11.14.65:39823)
WARNING: no policy specified for kudu/127.11.14.66@KRBTEST.COM; defaulting to no policy
I20260807 18:33:27.236548 15786 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.14.65@KRBTEST.COM'} at 127.11.14.65:52463
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.14.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.14.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.66.keytab.
Entry for principal kudu/127.11.14.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.66.keytab.
I20260807 18:33:27.259157 11321 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkz56XN/build/debug/bin/kudu
/tmp/dist-test-taskkz56XN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127579900523-11321-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127579900523-11321-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127579900523-11321-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127579900523-11321-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.14.66:0
--local_ip_for_outbound_sockets=127.11.14.66
--webserver_interface=127.11.14.66
--webserver_port=0
--tserver_master_addrs=127.11.14.126:38005
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.66.keytab
--principal=kudu/127.11.14.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.14.84:40365
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127579900523-11321-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127579900523-11321-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127579900523-11321-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:33:27.374415 15994 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:33:27.374730 15994 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:33:27.374814 15994 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:33:27.378949 15994 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:33:27.379040 15994 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:33:27.379137 15994 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.14.66
I20260807 18:33:27.384392 15994 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.14.84:40365
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127579900523-11321-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127579900523-11321-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.66.keytab
--principal=kudu/127.11.14.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.14.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127579900523-11321-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.14.66
--webserver_port=0
--tserver_master_addrs=127.11.14.126:38005
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15994
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.14.66
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127579900523-11321-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:33:27.385527 15994 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:33:27.386577 15994 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:33:27.396404 16002 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:33:27.396493 15994 server_base.cc:1034] running on GCE node
W20260807 18:33:27.396415 15999 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:33:27.396529 16000 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:33:27.397184 15994 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:33:27.397912 15994 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:33:27.399117 15994 hybrid_clock.cc:648] HybridClock initialized: now 1786127607399102 us; error 41 us; skew 500 ppm
Aug 07 18:33:27 dist-test-slave-6k22 krb5kdc[15739](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127607, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:33:27.402393 15994 init.cc:377] Logged in from keytab as kudu/127.11.14.66@KRBTEST.COM (short username kudu)
I20260807 18:33:27.403319 15994 fs_manager.cc:359] Metadata directory not provided
I20260807 18:33:27.403422 15994 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:33:27.403744 15994 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:33:27.405977 15994 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127579900523-11321-0/minicluster-data/ts-1/data/instance:
uuid: "079eba2df88c4e2a98e07af52f006b8c"
format_stamp: "Formatted at 2026-08-07 18:33:27 on dist-test-slave-6k22"
server_key: "6908f2c755dbd1754d7168f8ce7be576"
server_key_iv: "56ff634ac7b62c602fef7bae6a219d46"
server_key_version: "encryptionkey@0"
I20260807 18:33:27.406612 15994 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127579900523-11321-0/minicluster-data/ts-1/wal/instance:
uuid: "079eba2df88c4e2a98e07af52f006b8c"
format_stamp: "Formatted at 2026-08-07 18:33:27 on dist-test-slave-6k22"
server_key: "6908f2c755dbd1754d7168f8ce7be576"
server_key_iv: "56ff634ac7b62c602fef7bae6a219d46"
server_key_version: "encryptionkey@0"
I20260807 18:33:27.410682 15994 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 18:33:27.413295 16008 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:33:27.414714 15994 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 18:33:27.414860 15994 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127579900523-11321-0/minicluster-data/ts-1/data,/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127579900523-11321-0/minicluster-data/ts-1/wal
uuid: "079eba2df88c4e2a98e07af52f006b8c"
format_stamp: "Formatted at 2026-08-07 18:33:27 on dist-test-slave-6k22"
server_key: "6908f2c755dbd1754d7168f8ce7be576"
server_key_iv: "56ff634ac7b62c602fef7bae6a219d46"
server_key_version: "encryptionkey@0"
I20260807 18:33:27.414978 15994 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127579900523-11321-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127579900523-11321-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127579900523-11321-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:33:27.436666 15994 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:33:27.440457 15994 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:33:27.440690 15994 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:33:27.441355 15994 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:33:27.442289 15994 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:33:27.442368 15994 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:33:27.442449 15994 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:33:27.442502 15994 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:33:27.452435 15994 rpc_server.cc:307] RPC server started. Bound to: 127.11.14.66:34809
I20260807 18:33:27.452464 16132 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.14.66:34809 every 8 connection(s)
I20260807 18:33:27.453052 15994 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127579900523-11321-0/minicluster-data/ts-1/data/info.pb
I20260807 18:33:27.456827 11321 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkz56XN/build/debug/bin/kudu as pid 15994
I20260807 18:33:27.456979 11321 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127579900523-11321-0/minicluster-data/ts-1/wal/instance
I20260807 18:33:27.457302 11321 external_mini_cluster.cc:1428] Setting key 4322d8ed7ff1fb5f675b42d2e451cf5c
Aug 07 18:33:27 dist-test-slave-6k22 krb5kdc[15739](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127607, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.66@KRBTEST.COM for kudu/127.11.14.126@KRBTEST.COM
I20260807 18:33:27.468092 15828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:27.454834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.66:55695 (local address 127.11.14.126:38005)
0807 18:33:27.454981 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:27.454985 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:27.455977 (+ 992us) server_negotiation.cc:407] Connection header received
0807 18:33:27.457318 (+ 1341us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:27.457324 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:27.457403 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:27.457505 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:33:27.458867 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:27.459735 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:27.460920 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:27.461139 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:27.461250 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:27.463771 (+ 2521us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:33:27.463779 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:33:27.463786 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:33:27.463825 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:33:27.465761 (+ 1936us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:27.466277 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:27.466284 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:27.466289 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:27.466404 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:27.466757 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:27.466761 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:27.466763 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:27.467046 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:33:27.467171 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:27.467729 (+ 558us) server_negotiation.cc:299] Negotiation successful
0807 18:33:27.467878 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260807 18:33:27.468899 16136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:27.455308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:38005 (local address 127.11.14.66:55695)
0807 18:33:27.455804 (+ 496us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:27.455861 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:27.457096 (+ 1235us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:27.457676 (+ 580us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:27.457682 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:27.458144 (+ 462us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:33:27.458704 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:27.458724 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:27.459873 (+ 1149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:27.459876 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:27.460750 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:27.460772 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:27.461288 (+ 516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:27.461292 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:27.461463 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:27.462076 (+ 613us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:33:27.462093 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:33:27.463520 (+ 1427us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:33:27.465892 (+ 2372us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:33:27.465897 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:33:27.465909 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:33:27.466164 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:33:27.466509 (+ 345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:33:27.466511 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:33:27.466513 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:33:27.466650 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:33:27.467165 (+ 515us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:33:27.467171 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:33:27.467428 (+ 257us) client_negotiation.cc:769] Sending connection context
0807 18:33:27.467663 (+ 235us) client_negotiation.cc:240] Negotiation successful
0807 18:33:27.467938 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":301,"thread_start_us":116,"threads_started":1}
WARNING: no policy specified for kudu/127.11.14.67@KRBTEST.COM; defaulting to no policy
I20260807 18:33:27.470227 16133 heartbeater.cc:347] Connected to a master server at 127.11.14.126:38005
I20260807 18:33:27.470489 16133 heartbeater.cc:464] Registering TS with master...
I20260807 18:33:27.470969 16133 heartbeater.cc:511] Master 127.11.14.126:38005 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.14.67@KRBTEST.COM" created.
I20260807 18:33:27.472074 15786 ts_manager.cc:194] Registered new tserver with Master: 079eba2df88c4e2a98e07af52f006b8c (127.11.14.66:34809)
I20260807 18:33:27.472905 15786 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.14.66@KRBTEST.COM'} at 127.11.14.66:55695
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.14.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.67.keytab.
Entry for principal kudu/127.11.14.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.67.keytab.
I20260807 18:33:27.491549 11321 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkz56XN/build/debug/bin/kudu
/tmp/dist-test-taskkz56XN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127579900523-11321-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127579900523-11321-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127579900523-11321-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127579900523-11321-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.14.67:0
--local_ip_for_outbound_sockets=127.11.14.67
--webserver_interface=127.11.14.67
--webserver_port=0
--tserver_master_addrs=127.11.14.126:38005
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.67.keytab
--principal=kudu/127.11.14.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.14.84:40365
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127579900523-11321-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127579900523-11321-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127579900523-11321-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:33:27.610728 16139 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:33:27.610998 16139 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:33:27.611044 16139 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:33:27.614969 16139 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:33:27.615021 16139 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:33:27.615080 16139 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.14.67
I20260807 18:33:27.620617 16139 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.14.84:40365
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127579900523-11321-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127579900523-11321-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.67.keytab
--principal=kudu/127.11.14.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.14.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127579900523-11321-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.14.67
--webserver_port=0
--tserver_master_addrs=127.11.14.126:38005
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16139
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.14.67
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127579900523-11321-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:33:27.621637 16139 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:33:27.622664 16139 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:33:27.632305 16144 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:33:27.632489 16139 server_base.cc:1034] running on GCE node
W20260807 18:33:27.632308 16148 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:33:27.632294 16145 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:33:27.633181 16139 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:33:27.633814 16139 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:33:27.635003 16139 hybrid_clock.cc:648] HybridClock initialized: now 1786127607634983 us; error 31 us; skew 500 ppm
Aug 07 18:33:27 dist-test-slave-6k22 krb5kdc[15739](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127607, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:33:27.638208 16139 init.cc:377] Logged in from keytab as kudu/127.11.14.67@KRBTEST.COM (short username kudu)
I20260807 18:33:27.639037 16139 fs_manager.cc:359] Metadata directory not provided
I20260807 18:33:27.639135 16139 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:33:27.639389 16139 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:33:27.641610 16139 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127579900523-11321-0/minicluster-data/ts-2/data/instance:
uuid: "76bb34f858b64999bde0d6e5eb650ffa"
format_stamp: "Formatted at 2026-08-07 18:33:27 on dist-test-slave-6k22"
server_key: "7468aace0ac5c526a633f80eb6e6f0f7"
server_key_iv: "8c3fcf2cd07fcd9e3d5f3fbfd2a5115f"
server_key_version: "encryptionkey@0"
I20260807 18:33:27.642212 16139 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127579900523-11321-0/minicluster-data/ts-2/wal/instance:
uuid: "76bb34f858b64999bde0d6e5eb650ffa"
format_stamp: "Formatted at 2026-08-07 18:33:27 on dist-test-slave-6k22"
server_key: "7468aace0ac5c526a633f80eb6e6f0f7"
server_key_iv: "8c3fcf2cd07fcd9e3d5f3fbfd2a5115f"
server_key_version: "encryptionkey@0"
I20260807 18:33:27.646277 16139 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260807 18:33:27.648965 16155 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:33:27.650408 16139 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 18:33:27.650573 16139 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127579900523-11321-0/minicluster-data/ts-2/data,/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127579900523-11321-0/minicluster-data/ts-2/wal
uuid: "76bb34f858b64999bde0d6e5eb650ffa"
format_stamp: "Formatted at 2026-08-07 18:33:27 on dist-test-slave-6k22"
server_key: "7468aace0ac5c526a633f80eb6e6f0f7"
server_key_iv: "8c3fcf2cd07fcd9e3d5f3fbfd2a5115f"
server_key_version: "encryptionkey@0"
I20260807 18:33:27.650751 16139 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127579900523-11321-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127579900523-11321-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127579900523-11321-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:33:27.666606 16139 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:33:27.669973 16139 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:33:27.670182 16139 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:33:27.670784 16139 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:33:27.671686 16139 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:33:27.671761 16139 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:33:27.671836 16139 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:33:27.671869 16139 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:33:27.682032 16139 rpc_server.cc:307] RPC server started. Bound to: 127.11.14.67:45147
I20260807 18:33:27.682047 16277 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.14.67:45147 every 8 connection(s)
I20260807 18:33:27.682673 16139 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127579900523-11321-0/minicluster-data/ts-2/data/info.pb
I20260807 18:33:27.689482 11321 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkz56XN/build/debug/bin/kudu as pid 16139
I20260807 18:33:27.689631 11321 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127579900523-11321-0/minicluster-data/ts-2/wal/instance
I20260807 18:33:27.689956 11321 external_mini_cluster.cc:1428] Setting key 5e4280e420efef0c8c19d2249cccdadd
Aug 07 18:33:27 dist-test-slave-6k22 krb5kdc[15739](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127607, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.67@KRBTEST.COM for kudu/127.11.14.126@KRBTEST.COM
I20260807 18:33:27.697947 15828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:27.684709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.67:55507 (local address 127.11.14.126:38005)
0807 18:33:27.684892 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:27.684897 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:27.685772 (+ 875us) server_negotiation.cc:407] Connection header received
0807 18:33:27.687136 (+ 1364us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:27.687140 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:27.687188 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:27.687265 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:33:27.688705 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:27.689270 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:27.690951 (+ 1681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:27.691306 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:27.691436 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:27.694165 (+ 2729us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:33:27.694170 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:33:27.694173 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:33:27.694205 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:33:27.695678 (+ 1473us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:27.696258 (+ 580us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:27.696263 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:27.696265 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:27.696368 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:27.696748 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:27.696754 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:27.696756 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:27.696964 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:33:27.697086 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:27.697612 (+ 526us) server_negotiation.cc:299] Negotiation successful
0807 18:33:27.697753 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260807 18:33:27.698863 16280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:27.685090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:38005 (local address 127.11.14.67:55507)
0807 18:33:27.685602 (+ 512us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:27.685656 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:27.686920 (+ 1264us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:27.687476 (+ 556us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:27.687483 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:27.687975 (+ 492us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:33:27.688496 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:27.688510 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:27.689418 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:27.689422 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:27.690823 (+ 1401us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:27.690833 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:27.691478 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:27.691482 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:27.691794 (+ 312us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:27.692498 (+ 704us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:33:27.692515 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:33:27.693993 (+ 1478us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:33:27.695804 (+ 1811us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:33:27.695811 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:33:27.695827 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:33:27.696139 (+ 312us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:33:27.696472 (+ 333us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:33:27.696477 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:33:27.696479 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:33:27.696640 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:33:27.697084 (+ 444us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:33:27.697092 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:33:27.697348 (+ 256us) client_negotiation.cc:769] Sending connection context
0807 18:33:27.697549 (+ 201us) client_negotiation.cc:240] Negotiation successful
0807 18:33:27.697914 (+ 365us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":312,"thread_start_us":121,"threads_started":1}
I20260807 18:33:27.700289 16278 heartbeater.cc:347] Connected to a master server at 127.11.14.126:38005
I20260807 18:33:27.700573 16278 heartbeater.cc:464] Registering TS with master...
I20260807 18:33:27.701133 16278 heartbeater.cc:511] Master 127.11.14.126:38005 requested a full tablet report, sending...
I20260807 18:33:27.702343 15786 ts_manager.cc:194] Registered new tserver with Master: 76bb34f858b64999bde0d6e5eb650ffa (127.11.14.67:45147)
I20260807 18:33:27.703155 15786 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.14.67@KRBTEST.COM'} at 127.11.14.67:55507
I20260807 18:33:27.704504 11321 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 18:33:27.714797 15828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:27.706780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33064 (local address 127.11.14.126:38005)
0807 18:33:27.706956 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:27.706961 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:27.707136 (+ 175us) server_negotiation.cc:407] Connection header received
0807 18:33:27.707323 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:27.707326 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:27.707373 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:27.707475 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:33:27.708356 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:27.708880 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:27.709774 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:27.710015 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:27.710154 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:27.711544 (+ 1390us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:33:27.711548 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:33:27.711551 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:33:27.711602 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:33:27.712918 (+ 1316us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:27.713380 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:27.713386 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:27.713388 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:27.713483 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:27.713793 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:27.713798 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:27.713800 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:27.714033 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:33:27.714195 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:27.714455 (+ 260us) server_negotiation.cc:299] Negotiation successful
0807 18:33:27.714586 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260807 18:33:27.717751 15786 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33064:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 18:33:27.720888 15786 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 18:33:27.736186 16289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:27.729437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.66:34809 (local address 127.0.0.1:55858)
0807 18:33:27.730625 (+ 1188us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:27.730682 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:27.730826 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:27.731545 (+ 719us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:27.731601 (+ 56us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:27.731626 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:33:27.731817 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:27.731823 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:27.733396 (+ 1573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:27.733402 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:27.734514 (+ 1112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:27.734524 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:27.735605 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:27.735610 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:27.735725 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:27.735770 (+ 45us) client_negotiation.cc:769] Sending connection context
0807 18:33:27.735876 (+ 106us) client_negotiation.cc:240] Negotiation successful
0807 18:33:27.735999 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1002,"mutex_wait_us":50,"thread_start_us":95,"threads_started":1}
I20260807 18:33:27.736752 16294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:27.730311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41192 (local address 127.11.14.67:45147)
0807 18:33:27.731033 (+ 722us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:27.731044 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:27.731315 (+ 271us) server_negotiation.cc:407] Connection header received
0807 18:33:27.731426 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:27.731433 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:27.731632 (+ 199us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:27.731780 (+ 148us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:33:27.732375 (+ 595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:27.733658 (+ 1283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:27.735091 (+ 1433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:27.735764 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:27.735885 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:27.736025 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:27.736472 (+ 447us) server_negotiation.cc:299] Negotiation successful
0807 18:33:27.736571 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":631,"thread_start_us":161,"threads_started":1}
I20260807 18:33:27.736891 16293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:27.729439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.65:39823 (local address 127.0.0.1:42788)
0807 18:33:27.730667 (+ 1228us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:27.730682 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:27.730843 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:27.731824 (+ 981us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:27.731827 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:27.731846 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:33:27.731996 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:27.732007 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:27.733481 (+ 1474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:27.733484 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:27.734932 (+ 1448us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:27.734944 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:27.736482 (+ 1538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:27.736485 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:27.736567 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:27.736588 (+ 21us) client_negotiation.cc:769] Sending connection context
0807 18:33:27.736674 (+ 86us) client_negotiation.cc:240] Negotiation successful
0807 18:33:27.736734 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1062,"mutex_wait_us":176,"spinlock_wait_cycles":2944,"thread_start_us":109,"threads_started":1}
I20260807 18:33:27.737105 16291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:27.729554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55858 (local address 127.11.14.66:34809)
0807 18:33:27.730838 (+ 1284us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:27.730852 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:27.730880 (+ 28us) server_negotiation.cc:407] Connection header received
0807 18:33:27.730966 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:27.730971 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:27.731244 (+ 273us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:27.731536 (+ 292us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:33:27.731951 (+ 415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:27.733239 (+ 1288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:27.734654 (+ 1415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:27.735411 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:27.735550 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:27.735756 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:27.736830 (+ 1074us) server_negotiation.cc:299] Negotiation successful
0807 18:33:27.736930 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1160,"thread_start_us":90,"threads_started":1}
I20260807 18:33:27.736367 16290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:27.730088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.67:45147 (local address 127.0.0.1:41192)
0807 18:33:27.731084 (+ 996us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:27.731159 (+ 75us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:27.731308 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:27.731855 (+ 547us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:27.731858 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:27.731876 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:33:27.731999 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:27.732004 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:27.733801 (+ 1797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:27.733804 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:27.734959 (+ 1155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:27.734967 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:27.735895 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:27.735899 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:27.736005 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:27.736061 (+ 56us) client_negotiation.cc:769] Sending connection context
0807 18:33:27.736141 (+ 80us) client_negotiation.cc:240] Negotiation successful
0807 18:33:27.736223 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":933,"mutex_wait_us":28,"thread_start_us":116,"threads_started":1}
I20260807 18:33:27.737366 16292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:27.729737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42788 (local address 127.11.14.65:39823)
0807 18:33:27.730367 (+ 630us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:27.730376 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:27.730780 (+ 404us) server_negotiation.cc:407] Connection header received
0807 18:33:27.731022 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:27.731028 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:27.731467 (+ 439us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:27.731631 (+ 164us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:33:27.732129 (+ 498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:27.733363 (+ 1234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:27.735075 (+ 1712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:27.736011 (+ 936us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:27.736281 (+ 270us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:27.736423 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:27.737125 (+ 702us) server_negotiation.cc:299] Negotiation successful
0807 18:33:27.737226 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":505,"thread_start_us":134,"threads_started":1}
I20260807 18:33:27.740229 16062 tablet_service.cc:1467] Processing CreateTablet for tablet 8d1b0a9b75734ff3bc0f4f9cf708b4a2 (DEFAULT_TABLE table=test-table [id=4f87e73cb7194069a5f82a576d78b826]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:33:27.740226 15918 tablet_service.cc:1467] Processing CreateTablet for tablet 8d1b0a9b75734ff3bc0f4f9cf708b4a2 (DEFAULT_TABLE table=test-table [id=4f87e73cb7194069a5f82a576d78b826]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:33:27.741417 16062 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8d1b0a9b75734ff3bc0f4f9cf708b4a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:33:27.741417 15918 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8d1b0a9b75734ff3bc0f4f9cf708b4a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:33:27.742112 16207 tablet_service.cc:1467] Processing CreateTablet for tablet 8d1b0a9b75734ff3bc0f4f9cf708b4a2 (DEFAULT_TABLE table=test-table [id=4f87e73cb7194069a5f82a576d78b826]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:33:27.743023 16207 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8d1b0a9b75734ff3bc0f4f9cf708b4a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:33:27.749034 16295 tablet_bootstrap.cc:492] T 8d1b0a9b75734ff3bc0f4f9cf708b4a2 P 215f5426ceae453d982d600d5e9ee2ab: Bootstrap starting.
I20260807 18:33:27.749868 16296 tablet_bootstrap.cc:492] T 8d1b0a9b75734ff3bc0f4f9cf708b4a2 P 079eba2df88c4e2a98e07af52f006b8c: Bootstrap starting.
I20260807 18:33:27.750169 16297 tablet_bootstrap.cc:492] T 8d1b0a9b75734ff3bc0f4f9cf708b4a2 P 76bb34f858b64999bde0d6e5eb650ffa: Bootstrap starting.
I20260807 18:33:27.752130 16295 tablet_bootstrap.cc:654] T 8d1b0a9b75734ff3bc0f4f9cf708b4a2 P 215f5426ceae453d982d600d5e9ee2ab: Neither blocks nor log segments found. Creating new log.
I20260807 18:33:27.752131 16297 tablet_bootstrap.cc:654] T 8d1b0a9b75734ff3bc0f4f9cf708b4a2 P 76bb34f858b64999bde0d6e5eb650ffa: Neither blocks nor log segments found. Creating new log.
I20260807 18:33:27.752440 16296 tablet_bootstrap.cc:654] T 8d1b0a9b75734ff3bc0f4f9cf708b4a2 P 079eba2df88c4e2a98e07af52f006b8c: Neither blocks nor log segments found. Creating new log.
I20260807 18:33:27.753115 16296 log.cc:824] T 8d1b0a9b75734ff3bc0f4f9cf708b4a2 P 079eba2df88c4e2a98e07af52f006b8c: Log is configured to *not* fsync() on all Append() calls
I20260807 18:33:27.753130 16295 log.cc:824] T 8d1b0a9b75734ff3bc0f4f9cf708b4a2 P 215f5426ceae453d982d600d5e9ee2ab: Log is configured to *not* fsync() on all Append() calls
I20260807 18:33:27.753331 16297 log.cc:824] T 8d1b0a9b75734ff3bc0f4f9cf708b4a2 P 76bb34f858b64999bde0d6e5eb650ffa: Log is configured to *not* fsync() on all Append() calls
I20260807 18:33:27.755148 16296 tablet_bootstrap.cc:492] T 8d1b0a9b75734ff3bc0f4f9cf708b4a2 P 079eba2df88c4e2a98e07af52f006b8c: No bootstrap required, opened a new log
I20260807 18:33:27.755158 16295 tablet_bootstrap.cc:492] T 8d1b0a9b75734ff3bc0f4f9cf708b4a2 P 215f5426ceae453d982d600d5e9ee2ab: No bootstrap required, opened a new log
I20260807 18:33:27.755349 16296 ts_tablet_manager.cc:1397] T 8d1b0a9b75734ff3bc0f4f9cf708b4a2 P 079eba2df88c4e2a98e07af52f006b8c: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260807 18:33:27.755362 16295 ts_tablet_manager.cc:1397] T 8d1b0a9b75734ff3bc0f4f9cf708b4a2 P 215f5426ceae453d982d600d5e9ee2ab: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260807 18:33:27.756023 16297 tablet_bootstrap.cc:492] T 8d1b0a9b75734ff3bc0f4f9cf708b4a2 P 76bb34f858b64999bde0d6e5eb650ffa: No bootstrap required, opened a new log
I20260807 18:33:27.756208 16297 ts_tablet_manager.cc:1397] T 8d1b0a9b75734ff3bc0f4f9cf708b4a2 P 76bb34f858b64999bde0d6e5eb650ffa: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260807 18:33:27.759263 16295 raft_consensus.cc:348] T 8d1b0a9b75734ff3bc0f4f9cf708b4a2 P 215f5426ceae453d982d600d5e9ee2ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "079eba2df88c4e2a98e07af52f006b8c" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 34809 } } peers { permanent_uuid: "215f5426ceae453d982d600d5e9ee2ab" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 39823 } } peers { permanent_uuid: "76bb34f858b64999bde0d6e5eb650ffa" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 45147 } }
I20260807 18:33:27.759289 16296 raft_consensus.cc:348] T 8d1b0a9b75734ff3bc0f4f9cf708b4a2 P 079eba2df88c4e2a98e07af52f006b8c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "079eba2df88c4e2a98e07af52f006b8c" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 34809 } } peers { permanent_uuid: "215f5426ceae453d982d600d5e9ee2ab" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 39823 } } peers { permanent_uuid: "76bb34f858b64999bde0d6e5eb650ffa" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 45147 } }
I20260807 18:33:27.759501 16295 raft_consensus.cc:374] T 8d1b0a9b75734ff3bc0f4f9cf708b4a2 P 215f5426ceae453d982d600d5e9ee2ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:33:27.759558 16295 raft_consensus.cc:729] T 8d1b0a9b75734ff3bc0f4f9cf708b4a2 P 215f5426ceae453d982d600d5e9ee2ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 215f5426ceae453d982d600d5e9ee2ab, State: Initialized, Role: FOLLOWER
I20260807 18:33:27.759610 16296 raft_consensus.cc:374] T 8d1b0a9b75734ff3bc0f4f9cf708b4a2 P 079eba2df88c4e2a98e07af52f006b8c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:33:27.759680 16296 raft_consensus.cc:729] T 8d1b0a9b75734ff3bc0f4f9cf708b4a2 P 079eba2df88c4e2a98e07af52f006b8c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 079eba2df88c4e2a98e07af52f006b8c, State: Initialized, Role: FOLLOWER
I20260807 18:33:27.759944 16297 raft_consensus.cc:348] T 8d1b0a9b75734ff3bc0f4f9cf708b4a2 P 76bb34f858b64999bde0d6e5eb650ffa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "079eba2df88c4e2a98e07af52f006b8c" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 34809 } } peers { permanent_uuid: "215f5426ceae453d982d600d5e9ee2ab" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 39823 } } peers { permanent_uuid: "76bb34f858b64999bde0d6e5eb650ffa" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 45147 } }
I20260807 18:33:27.760076 16295 consensus_queue.cc:260] T 8d1b0a9b75734ff3bc0f4f9cf708b4a2 P 215f5426ceae453d982d600d5e9ee2ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "079eba2df88c4e2a98e07af52f006b8c" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 34809 } } peers { permanent_uuid: "215f5426ceae453d982d600d5e9ee2ab" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 39823 } } peers { permanent_uuid: "76bb34f858b64999bde0d6e5eb650ffa" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 45147 } }
I20260807 18:33:27.760200 16297 raft_consensus.cc:374] T 8d1b0a9b75734ff3bc0f4f9cf708b4a2 P 76bb34f858b64999bde0d6e5eb650ffa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:33:27.760164 16296 consensus_queue.cc:260] T 8d1b0a9b75734ff3bc0f4f9cf708b4a2 P 079eba2df88c4e2a98e07af52f006b8c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "079eba2df88c4e2a98e07af52f006b8c" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 34809 } } peers { permanent_uuid: "215f5426ceae453d982d600d5e9ee2ab" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 39823 } } peers { permanent_uuid: "76bb34f858b64999bde0d6e5eb650ffa" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 45147 } }
I20260807 18:33:27.760277 16297 raft_consensus.cc:729] T 8d1b0a9b75734ff3bc0f4f9cf708b4a2 P 76bb34f858b64999bde0d6e5eb650ffa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76bb34f858b64999bde0d6e5eb650ffa, State: Initialized, Role: FOLLOWER
I20260807 18:33:27.760706 16297 consensus_queue.cc:260] T 8d1b0a9b75734ff3bc0f4f9cf708b4a2 P 76bb34f858b64999bde0d6e5eb650ffa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "079eba2df88c4e2a98e07af52f006b8c" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 34809 } } peers { permanent_uuid: "215f5426ceae453d982d600d5e9ee2ab" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 39823 } } peers { permanent_uuid: "76bb34f858b64999bde0d6e5eb650ffa" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 45147 } }
I20260807 18:33:27.760787 15988 heartbeater.cc:503] Master 127.11.14.126:38005 was elected leader, sending a full tablet report...
I20260807 18:33:27.761086 16295 ts_tablet_manager.cc:1428] T 8d1b0a9b75734ff3bc0f4f9cf708b4a2 P 215f5426ceae453d982d600d5e9ee2ab: Time spent starting tablet: real 0.006s user 0.003s sys 0.004s
I20260807 18:33:27.761158 16296 ts_tablet_manager.cc:1428] T 8d1b0a9b75734ff3bc0f4f9cf708b4a2 P 079eba2df88c4e2a98e07af52f006b8c: Time spent starting tablet: real 0.006s user 0.001s sys 0.005s
I20260807 18:33:27.761535 16133 heartbeater.cc:503] Master 127.11.14.126:38005 was elected leader, sending a full tablet report...
I20260807 18:33:27.761694 16297 ts_tablet_manager.cc:1428] T 8d1b0a9b75734ff3bc0f4f9cf708b4a2 P 76bb34f858b64999bde0d6e5eb650ffa: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260807 18:33:27.762043 16278 heartbeater.cc:503] Master 127.11.14.126:38005 was elected leader, sending a full tablet report...
W20260807 18:33:27.933674 16279 tablet.cc:2369] T 8d1b0a9b75734ff3bc0f4f9cf708b4a2 P 76bb34f858b64999bde0d6e5eb650ffa: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 18:33:27.954365 16134 tablet.cc:2369] T 8d1b0a9b75734ff3bc0f4f9cf708b4a2 P 079eba2df88c4e2a98e07af52f006b8c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 18:33:27.966959 15989 tablet.cc:2369] T 8d1b0a9b75734ff3bc0f4f9cf708b4a2 P 215f5426ceae453d982d600d5e9ee2ab: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:33:28.036674 16303 raft_consensus.cc:482] T 8d1b0a9b75734ff3bc0f4f9cf708b4a2 P 76bb34f858b64999bde0d6e5eb650ffa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:33:28.036911 16303 raft_consensus.cc:504] T 8d1b0a9b75734ff3bc0f4f9cf708b4a2 P 76bb34f858b64999bde0d6e5eb650ffa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "079eba2df88c4e2a98e07af52f006b8c" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 34809 } } peers { permanent_uuid: "215f5426ceae453d982d600d5e9ee2ab" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 39823 } } peers { permanent_uuid: "76bb34f858b64999bde0d6e5eb650ffa" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 45147 } }
I20260807 18:33:28.038182 16303 leader_election.cc:288] T 8d1b0a9b75734ff3bc0f4f9cf708b4a2 P 76bb34f858b64999bde0d6e5eb650ffa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 079eba2df88c4e2a98e07af52f006b8c (127.11.14.66:34809), 215f5426ceae453d982d600d5e9ee2ab (127.11.14.65:39823)
I20260807 18:33:28.043308 16291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:28.038614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.67:54087 (local address 127.11.14.66:34809)
0807 18:33:28.038759 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:28.038764 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:28.039149 (+ 385us) server_negotiation.cc:407] Connection header received
0807 18:33:28.039240 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:28.039242 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:28.039284 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:28.039375 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:33:28.039645 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:28.040365 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:28.041947 (+ 1582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:28.042569 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:28.042743 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:28.042799 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:28.043094 (+ 295us) server_negotiation.cc:299] Negotiation successful
0807 18:33:28.043178 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260807 18:33:28.043308 16292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:28.038689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.67:54529 (local address 127.11.14.65:39823)
0807 18:33:28.038839 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:28.038844 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:28.038861 (+ 17us) server_negotiation.cc:407] Connection header received
0807 18:33:28.038931 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:28.038934 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:28.038978 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:28.039049 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:33:28.039399 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:28.040229 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:28.041947 (+ 1718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:28.042569 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:28.042744 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:28.042799 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:28.043094 (+ 295us) server_negotiation.cc:299] Negotiation successful
0807 18:33:28.043169 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260807 18:33:28.043362 16280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:28.038485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.65:39823 (local address 127.11.14.67:54529)
0807 18:33:28.038640 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:28.038658 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:28.038771 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:28.039098 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:28.039101 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:28.039128 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:33:28.039278 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:28.039284 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:28.040343 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:28.040345 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:28.041801 (+ 1456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:28.041807 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:28.042760 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:28.042764 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:28.042925 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:28.042979 (+ 54us) client_negotiation.cc:769] Sending connection context
0807 18:33:28.043062 (+ 83us) client_negotiation.cc:240] Negotiation successful
0807 18:33:28.043186 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57,"mutex_wait_us":79,"spinlock_wait_cycles":384}
I20260807 18:33:28.043375 16304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:28.038485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.66:34809 (local address 127.11.14.67:54087)
0807 18:33:28.039008 (+ 523us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:28.039019 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:28.039157 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:28.039388 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:28.039390 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:28.039414 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:33:28.039501 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:28.039505 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:28.040465 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:28.040468 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:28.041792 (+ 1324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:28.041805 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:28.042760 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:28.042764 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:28.042938 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:28.042977 (+ 39us) client_negotiation.cc:769] Sending connection context
0807 18:33:28.043062 (+ 85us) client_negotiation.cc:240] Negotiation successful
0807 18:33:28.043191 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":472,"mutex_wait_us":296,"spinlock_wait_cycles":6656,"thread_start_us":127,"threads_started":1}
I20260807 18:33:28.044083 15939 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d1b0a9b75734ff3bc0f4f9cf708b4a2" candidate_uuid: "76bb34f858b64999bde0d6e5eb650ffa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "215f5426ceae453d982d600d5e9ee2ab" is_pre_election: true
I20260807 18:33:28.044129 16084 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d1b0a9b75734ff3bc0f4f9cf708b4a2" candidate_uuid: "76bb34f858b64999bde0d6e5eb650ffa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "079eba2df88c4e2a98e07af52f006b8c" is_pre_election: true
I20260807 18:33:28.044457 16084 raft_consensus.cc:2454] T 8d1b0a9b75734ff3bc0f4f9cf708b4a2 P 079eba2df88c4e2a98e07af52f006b8c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 76bb34f858b64999bde0d6e5eb650ffa in term 0.
I20260807 18:33:28.044457 15939 raft_consensus.cc:2454] T 8d1b0a9b75734ff3bc0f4f9cf708b4a2 P 215f5426ceae453d982d600d5e9ee2ab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 76bb34f858b64999bde0d6e5eb650ffa in term 0.
I20260807 18:33:28.045025 16158 leader_election.cc:302] T 8d1b0a9b75734ff3bc0f4f9cf708b4a2 P 76bb34f858b64999bde0d6e5eb650ffa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 215f5426ceae453d982d600d5e9ee2ab, 76bb34f858b64999bde0d6e5eb650ffa; no voters:
I20260807 18:33:28.045367 16303 raft_consensus.cc:2781] T 8d1b0a9b75734ff3bc0f4f9cf708b4a2 P 76bb34f858b64999bde0d6e5eb650ffa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:33:28.045454 16303 raft_consensus.cc:482] T 8d1b0a9b75734ff3bc0f4f9cf708b4a2 P 76bb34f858b64999bde0d6e5eb650ffa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:33:28.045506 16303 raft_consensus.cc:3037] T 8d1b0a9b75734ff3bc0f4f9cf708b4a2 P 76bb34f858b64999bde0d6e5eb650ffa [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:33:28.046674 16303 raft_consensus.cc:504] T 8d1b0a9b75734ff3bc0f4f9cf708b4a2 P 76bb34f858b64999bde0d6e5eb650ffa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "079eba2df88c4e2a98e07af52f006b8c" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 34809 } } peers { permanent_uuid: "215f5426ceae453d982d600d5e9ee2ab" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 39823 } } peers { permanent_uuid: "76bb34f858b64999bde0d6e5eb650ffa" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 45147 } }
I20260807 18:33:28.047031 16303 leader_election.cc:288] T 8d1b0a9b75734ff3bc0f4f9cf708b4a2 P 76bb34f858b64999bde0d6e5eb650ffa [CANDIDATE]: Term 1 election: Requested vote from peers 079eba2df88c4e2a98e07af52f006b8c (127.11.14.66:34809), 215f5426ceae453d982d600d5e9ee2ab (127.11.14.65:39823)
I20260807 18:33:28.047495 16084 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d1b0a9b75734ff3bc0f4f9cf708b4a2" candidate_uuid: "76bb34f858b64999bde0d6e5eb650ffa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "079eba2df88c4e2a98e07af52f006b8c"
I20260807 18:33:28.047495 15939 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d1b0a9b75734ff3bc0f4f9cf708b4a2" candidate_uuid: "76bb34f858b64999bde0d6e5eb650ffa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "215f5426ceae453d982d600d5e9ee2ab"
I20260807 18:33:28.047655 16084 raft_consensus.cc:3037] T 8d1b0a9b75734ff3bc0f4f9cf708b4a2 P 079eba2df88c4e2a98e07af52f006b8c [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:33:28.047699 15939 raft_consensus.cc:3037] T 8d1b0a9b75734ff3bc0f4f9cf708b4a2 P 215f5426ceae453d982d600d5e9ee2ab [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:33:28.049042 16084 raft_consensus.cc:2454] T 8d1b0a9b75734ff3bc0f4f9cf708b4a2 P 079eba2df88c4e2a98e07af52f006b8c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 76bb34f858b64999bde0d6e5eb650ffa in term 1.
I20260807 18:33:28.049042 15939 raft_consensus.cc:2454] T 8d1b0a9b75734ff3bc0f4f9cf708b4a2 P 215f5426ceae453d982d600d5e9ee2ab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 76bb34f858b64999bde0d6e5eb650ffa in term 1.
I20260807 18:33:28.049433 16159 leader_election.cc:302] T 8d1b0a9b75734ff3bc0f4f9cf708b4a2 P 76bb34f858b64999bde0d6e5eb650ffa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 079eba2df88c4e2a98e07af52f006b8c, 76bb34f858b64999bde0d6e5eb650ffa; no voters:
I20260807 18:33:28.049671 16303 raft_consensus.cc:2781] T 8d1b0a9b75734ff3bc0f4f9cf708b4a2 P 76bb34f858b64999bde0d6e5eb650ffa [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:33:28.049971 16303 raft_consensus.cc:686] T 8d1b0a9b75734ff3bc0f4f9cf708b4a2 P 76bb34f858b64999bde0d6e5eb650ffa [term 1 LEADER]: Becoming Leader. State: Replica: 76bb34f858b64999bde0d6e5eb650ffa, State: Running, Role: LEADER
I20260807 18:33:28.050315 16303 consensus_queue.cc:237] T 8d1b0a9b75734ff3bc0f4f9cf708b4a2 P 76bb34f858b64999bde0d6e5eb650ffa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "079eba2df88c4e2a98e07af52f006b8c" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 34809 } } peers { permanent_uuid: "215f5426ceae453d982d600d5e9ee2ab" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 39823 } } peers { permanent_uuid: "76bb34f858b64999bde0d6e5eb650ffa" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 45147 } }
I20260807 18:33:28.053872 15785 catalog_manager.cc:5495] T 8d1b0a9b75734ff3bc0f4f9cf708b4a2 P 76bb34f858b64999bde0d6e5eb650ffa reported cstate change: term changed from 0 to 1, leader changed from <none> to 76bb34f858b64999bde0d6e5eb650ffa (127.11.14.67). New cstate: current_term: 1 leader_uuid: "76bb34f858b64999bde0d6e5eb650ffa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "079eba2df88c4e2a98e07af52f006b8c" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 34809 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "215f5426ceae453d982d600d5e9ee2ab" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 39823 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "76bb34f858b64999bde0d6e5eb650ffa" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 45147 } health_report { overall_health: HEALTHY } } }
I20260807 18:33:28.100387 16294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:28.096448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41208 (local address 127.11.14.67:45147)
0807 18:33:28.096588 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:28.096591 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:28.096606 (+ 15us) server_negotiation.cc:407] Connection header received
0807 18:33:28.096779 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:28.096782 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:28.096841 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:28.096946 (+ 105us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:33:28.097266 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:28.097822 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:28.098791 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:28.099011 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:28.099097 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:28.099431 (+ 334us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:33:28.099970 (+ 539us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:33:28.100069 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:28.100217 (+ 148us) server_negotiation.cc:299] Negotiation successful
0807 18:33:28.100274 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260807 18:33:28.106876 15939 raft_consensus.cc:1261] T 8d1b0a9b75734ff3bc0f4f9cf708b4a2 P 215f5426ceae453d982d600d5e9ee2ab [term 1 FOLLOWER]: Refusing update from remote peer 76bb34f858b64999bde0d6e5eb650ffa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:33:28.106877 16084 raft_consensus.cc:1261] T 8d1b0a9b75734ff3bc0f4f9cf708b4a2 P 079eba2df88c4e2a98e07af52f006b8c [term 1 FOLLOWER]: Refusing update from remote peer 76bb34f858b64999bde0d6e5eb650ffa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:33:28.107694 16305 consensus_queue.cc:1035] T 8d1b0a9b75734ff3bc0f4f9cf708b4a2 P 76bb34f858b64999bde0d6e5eb650ffa [LEADER]: Connected to new peer: Peer: permanent_uuid: "215f5426ceae453d982d600d5e9ee2ab" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 39823 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:33:28.107908 16303 consensus_queue.cc:1035] T 8d1b0a9b75734ff3bc0f4f9cf708b4a2 P 76bb34f858b64999bde0d6e5eb650ffa [LEADER]: Connected to new peer: Peer: permanent_uuid: "079eba2df88c4e2a98e07af52f006b8c" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 34809 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:33:28.114920 16310 mvcc.cc:205] Tried to move back new op lower bound from 7315978682797985792 to 7315978682576797696. Current Snapshot: MvccSnapshot[applied={T|T < 7315978682797985792}]
I20260807 18:33:28.115721 16312 mvcc.cc:205] Tried to move back new op lower bound from 7315978682797985792 to 7315978682576797696. Current Snapshot: MvccSnapshot[applied={T|T < 7315978682797985792}]
I20260807 18:33:28.116410 16311 mvcc.cc:205] Tried to move back new op lower bound from 7315978682797985792 to 7315978682576797696. Current Snapshot: MvccSnapshot[applied={T|T < 7315978682797985792}]
I20260807 18:33:28.124900 15784 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33064:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:33:28.125058 15784 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33064:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:33:28.127488 15784 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b53a559e96c74f7aa8e56f91a49ab0b8: Sending DeleteTablet for 3 replicas of tablet 8d1b0a9b75734ff3bc0f4f9cf708b4a2
I20260807 18:33:28.128353 16062 tablet_service.cc:1514] Processing DeleteTablet for tablet 8d1b0a9b75734ff3bc0f4f9cf708b4a2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:33:28 UTC) from {username='kudu', principal='kudu/127.11.14.126@KRBTEST.COM'} at 127.0.0.1:55858
I20260807 18:33:28.128472 15918 tablet_service.cc:1514] Processing DeleteTablet for tablet 8d1b0a9b75734ff3bc0f4f9cf708b4a2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:33:28 UTC) from {username='kudu', principal='kudu/127.11.14.126@KRBTEST.COM'} at 127.0.0.1:42788
I20260807 18:33:28.128468 16207 tablet_service.cc:1514] Processing DeleteTablet for tablet 8d1b0a9b75734ff3bc0f4f9cf708b4a2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:33:28 UTC) from {username='kudu', principal='kudu/127.11.14.126@KRBTEST.COM'} at 127.0.0.1:41192
I20260807 18:33:28.129052 16321 tablet_replica.cc:330] T 8d1b0a9b75734ff3bc0f4f9cf708b4a2 P 215f5426ceae453d982d600d5e9ee2ab: stopping tablet replica
I20260807 18:33:28.129318 16319 tablet_replica.cc:330] T 8d1b0a9b75734ff3bc0f4f9cf708b4a2 P 079eba2df88c4e2a98e07af52f006b8c: stopping tablet replica
I20260807 18:33:28.129411 16321 raft_consensus.cc:2229] T 8d1b0a9b75734ff3bc0f4f9cf708b4a2 P 215f5426ceae453d982d600d5e9ee2ab [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:33:28.129590 11321 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkz56XN/build/debug/bin/kudu with pid 15852
I20260807 18:33:28.132494 16319 raft_consensus.cc:2229] T 8d1b0a9b75734ff3bc0f4f9cf708b4a2 P 079eba2df88c4e2a98e07af52f006b8c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:33:28.132947 16319 raft_consensus.cc:2258] T 8d1b0a9b75734ff3bc0f4f9cf708b4a2 P 079eba2df88c4e2a98e07af52f006b8c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:33:28.135329 16319 ts_tablet_manager.cc:1905] T 8d1b0a9b75734ff3bc0f4f9cf708b4a2 P 079eba2df88c4e2a98e07af52f006b8c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:33:28.135782 16320 tablet_replica.cc:330] T 8d1b0a9b75734ff3bc0f4f9cf708b4a2 P 76bb34f858b64999bde0d6e5eb650ffa: stopping tablet replica
I20260807 18:33:28.136112 16320 raft_consensus.cc:2229] T 8d1b0a9b75734ff3bc0f4f9cf708b4a2 P 76bb34f858b64999bde0d6e5eb650ffa [term 1 LEADER]: Raft consensus shutting down.
I20260807 18:33:28.136595 16320 raft_consensus.cc:2258] T 8d1b0a9b75734ff3bc0f4f9cf708b4a2 P 76bb34f858b64999bde0d6e5eb650ffa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:33:28.138239 16319 ts_tablet_manager.cc:1918] T 8d1b0a9b75734ff3bc0f4f9cf708b4a2 P 079eba2df88c4e2a98e07af52f006b8c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:33:28.138417 16319 log.cc:1196] T 8d1b0a9b75734ff3bc0f4f9cf708b4a2 P 079eba2df88c4e2a98e07af52f006b8c: Deleting WAL directory at /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127579900523-11321-0/minicluster-data/ts-1/wal/wals/8d1b0a9b75734ff3bc0f4f9cf708b4a2
I20260807 18:33:28.138854 16319 ts_tablet_manager.cc:1939] T 8d1b0a9b75734ff3bc0f4f9cf708b4a2 P 079eba2df88c4e2a98e07af52f006b8c: Deleting consensus metadata
I20260807 18:33:28.138863 16320 ts_tablet_manager.cc:1905] T 8d1b0a9b75734ff3bc0f4f9cf708b4a2 P 76bb34f858b64999bde0d6e5eb650ffa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:33:28.140301 15772 catalog_manager.cc:4841] TS 079eba2df88c4e2a98e07af52f006b8c (127.11.14.66:34809): tablet 8d1b0a9b75734ff3bc0f4f9cf708b4a2 (table test-table [id=4f87e73cb7194069a5f82a576d78b826]) successfully deleted
I20260807 18:33:28.141307 16320 ts_tablet_manager.cc:1918] T 8d1b0a9b75734ff3bc0f4f9cf708b4a2 P 76bb34f858b64999bde0d6e5eb650ffa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:33:28.141426 16320 log.cc:1196] T 8d1b0a9b75734ff3bc0f4f9cf708b4a2 P 76bb34f858b64999bde0d6e5eb650ffa: Deleting WAL directory at /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786127579900523-11321-0/minicluster-data/ts-2/wal/wals/8d1b0a9b75734ff3bc0f4f9cf708b4a2
I20260807 18:33:28.141758 16320 ts_tablet_manager.cc:1939] T 8d1b0a9b75734ff3bc0f4f9cf708b4a2 P 76bb34f858b64999bde0d6e5eb650ffa: Deleting consensus metadata
W20260807 18:33:28.142482 15771 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.14.65:39823 (error 108)
I20260807 18:33:28.142892 15769 catalog_manager.cc:4841] TS 76bb34f858b64999bde0d6e5eb650ffa (127.11.14.67:45147): tablet 8d1b0a9b75734ff3bc0f4f9cf708b4a2 (table test-table [id=4f87e73cb7194069a5f82a576d78b826]) successfully deleted
I20260807 18:33:28.143704 11321 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkz56XN/build/debug/bin/kudu with pid 15994
I20260807 18:33:28.144141 16290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:28.143759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.65:39823 (local address 127.0.0.1:42790)
0807 18:33:28.143960 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:28.144038 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.65:39823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105}
W20260807 18:33:28.144354 15771 catalog_manager.cc:4584] TS 215f5426ceae453d982d600d5e9ee2ab (127.11.14.65:39823): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8d1b0a9b75734ff3bc0f4f9cf708b4a2: Network error: Client connection negotiation failed: client connection to 127.11.14.65:39823: connect: Connection refused (error 111)
I20260807 18:33:28.152514 11321 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkz56XN/build/debug/bin/kudu with pid 16139
I20260807 18:33:28.160845 11321 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkz56XN/build/debug/bin/kudu with pid 15753
2026-08-07T18:33:28Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2704 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-07T18:33:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:33:28Z Disabled control of system clock
I20260807 18:33:28.194271 11321 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkz56XN/build/debug/bin/kudu
/tmp/dist-test-taskkz56XN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786127579900523-11321-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786127579900523-11321-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786127579900523-11321-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786127579900523-11321-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.14.126:38827
--webserver_interface=127.11.14.126
--webserver_port=0
--builtin_ntp_servers=127.11.14.84:35233
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.14.126:38827
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260807 18:33:28.309508 16327 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:33:28.309818 16327 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:33:28.309856 16327 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:33:28.313905 16327 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:33:28.313972 16327 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:33:28.313988 16327 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:33:28.314004 16327 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:33:28.314019 16327 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:33:28.318866 16327 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.14.84:35233
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786127579900523-11321-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786127579900523-11321-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.14.126:38827
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.14.126:38827
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786127579900523-11321-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.14.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16327
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786127579900523-11321-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:33:28.320197 16327 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:33:28.321666 16327 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:33:28.330540 16333 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:33:28.330533 16332 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:33:28.330520 16335 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:33:28.330583 16327 server_base.cc:1034] running on GCE node
I20260807 18:33:28.331459 16327 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:33:28.332681 16327 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:33:28.333905 16327 hybrid_clock.cc:648] HybridClock initialized: now 1786127608333885 us; error 37 us; skew 500 ppm
I20260807 18:33:28.336258 16327 webserver.cc:460] Webserver started at http://127.11.14.126:36697/ using document root <none> and password file <none>
I20260807 18:33:28.336941 16327 fs_manager.cc:359] Metadata directory not provided
I20260807 18:33:28.337049 16327 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:33:28.337332 16327 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:33:28.339668 16327 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786127579900523-11321-0/minicluster-data/master-0/data/instance:
uuid: "fab2b364e6c0404f961b778da8c21f86"
format_stamp: "Formatted at 2026-08-07 18:33:28 on dist-test-slave-6k22"
server_key: "e174d7e7757304aac99281b074fad0fc"
server_key_iv: "f746bbce6a29936faebad556b40ea43a"
server_key_version: "encryptionkey@0"
I20260807 18:33:28.340372 16327 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786127579900523-11321-0/minicluster-data/master-0/wal/instance:
uuid: "fab2b364e6c0404f961b778da8c21f86"
format_stamp: "Formatted at 2026-08-07 18:33:28 on dist-test-slave-6k22"
server_key: "e174d7e7757304aac99281b074fad0fc"
server_key_iv: "f746bbce6a29936faebad556b40ea43a"
server_key_version: "encryptionkey@0"
I20260807 18:33:28.344645 16327 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260807 18:33:28.347203 16341 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:33:28.348502 16327 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.001s
I20260807 18:33:28.348650 16327 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786127579900523-11321-0/minicluster-data/master-0/data,/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786127579900523-11321-0/minicluster-data/master-0/wal
uuid: "fab2b364e6c0404f961b778da8c21f86"
format_stamp: "Formatted at 2026-08-07 18:33:28 on dist-test-slave-6k22"
server_key: "e174d7e7757304aac99281b074fad0fc"
server_key_iv: "f746bbce6a29936faebad556b40ea43a"
server_key_version: "encryptionkey@0"
I20260807 18:33:28.348784 16327 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786127579900523-11321-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786127579900523-11321-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786127579900523-11321-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:33:28.353504 16327 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:33:28.354122 16327 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:33:28.354319 16327 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:33:28.361549 16327 rpc_server.cc:307] RPC server started. Bound to: 127.11.14.126:38827
I20260807 18:33:28.361554 16398 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.14.126:38827 every 8 connection(s)
I20260807 18:33:28.362553 16327 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786127579900523-11321-0/minicluster-data/master-0/data/info.pb
I20260807 18:33:28.366158 16400 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:33:28.370620 11321 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkz56XN/build/debug/bin/kudu as pid 16327
I20260807 18:33:28.370736 11321 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786127579900523-11321-0/minicluster-data/master-0/wal/instance
I20260807 18:33:28.371054 11321 external_mini_cluster.cc:1428] Setting key cb5efdcd5f592e80e3b8ab9a5ed0fad6
I20260807 18:33:28.372238 16400 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fab2b364e6c0404f961b778da8c21f86: Bootstrap starting.
I20260807 18:33:28.374507 16400 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fab2b364e6c0404f961b778da8c21f86: Neither blocks nor log segments found. Creating new log.
I20260807 18:33:28.375536 16400 log.cc:824] T 00000000000000000000000000000000 P fab2b364e6c0404f961b778da8c21f86: Log is configured to *not* fsync() on all Append() calls
I20260807 18:33:28.377127 16404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:28.372639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51498 (local address 127.11.14.126:38827)
0807 18:33:28.373356 (+ 717us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:28.373373 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:28.373410 (+ 37us) server_negotiation.cc:407] Connection header received
0807 18:33:28.374065 (+ 655us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:28.374081 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:28.374446 (+ 365us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:28.374799 (+ 353us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:33:28.375723 (+ 924us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:33:28.375729 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:33:28.375751 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:33:28.375763 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:33:28.375819 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:33:28.375836 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:33:28.375929 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:28.376122 (+ 193us) server_negotiation.cc:299] Negotiation successful
0807 18:33:28.376396 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":284,"thread_start_us":144,"threads_started":1}
I20260807 18:33:28.377753 16400 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fab2b364e6c0404f961b778da8c21f86: No bootstrap required, opened a new log
I20260807 18:33:28.381726 16400 raft_consensus.cc:348] T 00000000000000000000000000000000 P fab2b364e6c0404f961b778da8c21f86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fab2b364e6c0404f961b778da8c21f86" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 38827 } }
I20260807 18:33:28.381980 16400 raft_consensus.cc:374] T 00000000000000000000000000000000 P fab2b364e6c0404f961b778da8c21f86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:33:28.382062 16400 raft_consensus.cc:729] T 00000000000000000000000000000000 P fab2b364e6c0404f961b778da8c21f86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fab2b364e6c0404f961b778da8c21f86, State: Initialized, Role: FOLLOWER
I20260807 18:33:28.382477 16400 consensus_queue.cc:260] T 00000000000000000000000000000000 P fab2b364e6c0404f961b778da8c21f86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fab2b364e6c0404f961b778da8c21f86" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 38827 } }
I20260807 18:33:28.382635 16400 raft_consensus.cc:388] T 00000000000000000000000000000000 P fab2b364e6c0404f961b778da8c21f86 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:33:28.382725 16400 raft_consensus.cc:482] T 00000000000000000000000000000000 P fab2b364e6c0404f961b778da8c21f86 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:33:28.382827 16400 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fab2b364e6c0404f961b778da8c21f86 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:33:28.383850 16400 raft_consensus.cc:504] T 00000000000000000000000000000000 P fab2b364e6c0404f961b778da8c21f86 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fab2b364e6c0404f961b778da8c21f86" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 38827 } }
I20260807 18:33:28.384191 16400 leader_election.cc:302] T 00000000000000000000000000000000 P fab2b364e6c0404f961b778da8c21f86 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fab2b364e6c0404f961b778da8c21f86; no voters:
I20260807 18:33:28.384516 16400 leader_election.cc:288] T 00000000000000000000000000000000 P fab2b364e6c0404f961b778da8c21f86 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:33:28.384672 16406 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fab2b364e6c0404f961b778da8c21f86 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:33:28.384922 16406 raft_consensus.cc:686] T 00000000000000000000000000000000 P fab2b364e6c0404f961b778da8c21f86 [term 1 LEADER]: Becoming Leader. State: Replica: fab2b364e6c0404f961b778da8c21f86, State: Running, Role: LEADER
I20260807 18:33:28.385286 16406 consensus_queue.cc:237] T 00000000000000000000000000000000 P fab2b364e6c0404f961b778da8c21f86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fab2b364e6c0404f961b778da8c21f86" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 38827 } }
I20260807 18:33:28.385922 16400 sys_catalog.cc:564] T 00000000000000000000000000000000 P fab2b364e6c0404f961b778da8c21f86 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:33:28.387068 16407 sys_catalog.cc:455] T 00000000000000000000000000000000 P fab2b364e6c0404f961b778da8c21f86 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fab2b364e6c0404f961b778da8c21f86. Latest consensus state: current_term: 1 leader_uuid: "fab2b364e6c0404f961b778da8c21f86" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fab2b364e6c0404f961b778da8c21f86" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 38827 } } }
I20260807 18:33:28.387117 16408 sys_catalog.cc:455] T 00000000000000000000000000000000 P fab2b364e6c0404f961b778da8c21f86 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fab2b364e6c0404f961b778da8c21f86" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fab2b364e6c0404f961b778da8c21f86" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 38827 } } }
I20260807 18:33:28.387243 16408 sys_catalog.cc:458] T 00000000000000000000000000000000 P fab2b364e6c0404f961b778da8c21f86 [sys.catalog]: This master's current role is: LEADER
I20260807 18:33:28.387253 16407 sys_catalog.cc:458] T 00000000000000000000000000000000 P fab2b364e6c0404f961b778da8c21f86 [sys.catalog]: This master's current role is: LEADER
I20260807 18:33:28.387681 16415 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:33:28.390772 16415 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:33:28.396566 16415 catalog_manager.cc:1347] Generated new cluster ID: d368e0598e3f4c59b11304b0020e6046
I20260807 18:33:28.396701 16415 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:33:28.415759 16415 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:33:28.416126 16415 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:33:28.429790 16415 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fab2b364e6c0404f961b778da8c21f86: Generated new TSK 0
I20260807 18:33:28.430672 16415 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:33:28.446336 11321 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkz56XN/build/debug/bin/kudu
/tmp/dist-test-taskkz56XN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786127579900523-11321-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786127579900523-11321-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786127579900523-11321-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786127579900523-11321-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.14.65:0
--local_ip_for_outbound_sockets=127.11.14.65
--webserver_interface=127.11.14.65
--webserver_port=0
--tserver_master_addrs=127.11.14.126:38827
--builtin_ntp_servers=127.11.14.84:35233
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260807 18:33:28.565375 16425 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:33:28.565632 16425 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:33:28.565680 16425 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:33:28.569868 16425 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:33:28.569932 16425 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:33:28.569998 16425 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.14.65
I20260807 18:33:28.574966 16425 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.14.84:35233
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786127579900523-11321-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786127579900523-11321-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.14.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786127579900523-11321-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.14.65
--webserver_port=0
--tserver_master_addrs=127.11.14.126:38827
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16425
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.14.65
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786127579900523-11321-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:33:28.576169 16425 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:33:28.577621 16425 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:33:28.587776 16431 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:33:28.587772 16433 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:33:28.587765 16430 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:33:28.588012 16425 server_base.cc:1034] running on GCE node
I20260807 18:33:28.588562 16425 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:33:28.589262 16425 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:33:28.590456 16425 hybrid_clock.cc:648] HybridClock initialized: now 1786127608590422 us; error 46 us; skew 500 ppm
I20260807 18:33:28.592355 16425 webserver.cc:460] Webserver started at http://127.11.14.65:44501/ using document root <none> and password file <none>
I20260807 18:33:28.592963 16425 fs_manager.cc:359] Metadata directory not provided
I20260807 18:33:28.593060 16425 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:33:28.593402 16425 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:33:28.595522 16425 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786127579900523-11321-0/minicluster-data/ts-0/data/instance:
uuid: "28b3d41c8dbf482e828645377eac64a0"
format_stamp: "Formatted at 2026-08-07 18:33:28 on dist-test-slave-6k22"
server_key: "ece8fbdb3b661e9891f9f0cd5a54737f"
server_key_iv: "83a53b1315148ca70ca981aaa9e98802"
server_key_version: "encryptionkey@0"
I20260807 18:33:28.596180 16425 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786127579900523-11321-0/minicluster-data/ts-0/wal/instance:
uuid: "28b3d41c8dbf482e828645377eac64a0"
format_stamp: "Formatted at 2026-08-07 18:33:28 on dist-test-slave-6k22"
server_key: "ece8fbdb3b661e9891f9f0cd5a54737f"
server_key_iv: "83a53b1315148ca70ca981aaa9e98802"
server_key_version: "encryptionkey@0"
I20260807 18:33:28.600025 16425 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 18:33:28.602697 16439 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:33:28.604009 16425 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 18:33:28.604157 16425 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786127579900523-11321-0/minicluster-data/ts-0/data,/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786127579900523-11321-0/minicluster-data/ts-0/wal
uuid: "28b3d41c8dbf482e828645377eac64a0"
format_stamp: "Formatted at 2026-08-07 18:33:28 on dist-test-slave-6k22"
server_key: "ece8fbdb3b661e9891f9f0cd5a54737f"
server_key_iv: "83a53b1315148ca70ca981aaa9e98802"
server_key_version: "encryptionkey@0"
I20260807 18:33:28.604290 16425 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786127579900523-11321-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786127579900523-11321-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786127579900523-11321-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:33:28.609061 16425 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:33:28.609661 16425 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:33:28.609807 16425 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:33:28.610308 16425 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:33:28.611192 16425 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:33:28.611239 16425 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:33:28.611286 16425 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:33:28.611318 16425 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:33:28.620649 16425 rpc_server.cc:307] RPC server started. Bound to: 127.11.14.65:37117
I20260807 18:33:28.620667 16561 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.14.65:37117 every 8 connection(s)
I20260807 18:33:28.621631 16425 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786127579900523-11321-0/minicluster-data/ts-0/data/info.pb
I20260807 18:33:28.624119 11321 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkz56XN/build/debug/bin/kudu as pid 16425
I20260807 18:33:28.624259 11321 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786127579900523-11321-0/minicluster-data/ts-0/wal/instance
I20260807 18:33:28.624616 11321 external_mini_cluster.cc:1428] Setting key c6c2d1f1114c34b2bbd3dae7707e5955
I20260807 18:33:28.626930 11321 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkz56XN/build/debug/bin/kudu
/tmp/dist-test-taskkz56XN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786127579900523-11321-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786127579900523-11321-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786127579900523-11321-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786127579900523-11321-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.14.66:0
--local_ip_for_outbound_sockets=127.11.14.66
--webserver_interface=127.11.14.66
--webserver_port=0
--tserver_master_addrs=127.11.14.126:38827
--builtin_ntp_servers=127.11.14.84:35233
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260807 18:33:28.628631 16404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:28.623521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.65:42509 (local address 127.11.14.126:38827)
0807 18:33:28.623727 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:28.623732 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:28.625066 (+ 1334us) server_negotiation.cc:407] Connection header received
0807 18:33:28.626711 (+ 1645us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:28.626714 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:28.626748 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:28.626823 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:33:28.628150 (+ 1327us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:33:28.628153 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:33:28.628155 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:33:28.628167 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:33:28.628189 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:33:28.628197 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:33:28.628272 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:28.628489 (+ 217us) server_negotiation.cc:299] Negotiation successful
0807 18:33:28.628527 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
I20260807 18:33:28.629583 16565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:28.624029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:38827 (local address 127.11.14.65:42509)
0807 18:33:28.624892 (+ 863us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:28.624949 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:28.626473 (+ 1524us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:28.627038 (+ 565us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:28.627051 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:28.627175 (+ 124us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:33:28.627827 (+ 652us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 18:33:28.627831 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:33:28.627890 (+ 59us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 18:33:28.627987 (+ 97us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:33:28.628307 (+ 320us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:33:28.628320 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:33:28.628329 (+ 9us) client_negotiation.cc:769] Sending connection context
0807 18:33:28.628518 (+ 189us) client_negotiation.cc:240] Negotiation successful
0807 18:33:28.628801 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":314,"thread_start_us":146,"threads_started":1}
I20260807 18:33:28.630995 16563 heartbeater.cc:347] Connected to a master server at 127.11.14.126:38827
I20260807 18:33:28.631237 16563 heartbeater.cc:464] Registering TS with master...
I20260807 18:33:28.631702 16563 heartbeater.cc:511] Master 127.11.14.126:38827 requested a full tablet report, sending...
I20260807 18:33:28.633541 16360 ts_manager.cc:194] Registered new tserver with Master: 28b3d41c8dbf482e828645377eac64a0 (127.11.14.65:37117)
W20260807 18:33:28.743199 16566 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:33:28.743465 16566 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:33:28.743505 16566 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:33:28.747638 16566 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:33:28.747701 16566 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:33:28.747758 16566 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.14.66
I20260807 18:33:28.753046 16566 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.14.84:35233
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786127579900523-11321-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786127579900523-11321-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.14.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786127579900523-11321-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.14.66
--webserver_port=0
--tserver_master_addrs=127.11.14.126:38827
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16566
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.14.66
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786127579900523-11321-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:33:28.754272 16566 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:33:28.755733 16566 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:33:28.765929 16572 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:33:28.765977 16574 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:33:28.765928 16571 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:33:28.766254 16566 server_base.cc:1034] running on GCE node
I20260807 18:33:28.766736 16566 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:33:28.767402 16566 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:33:28.768627 16566 hybrid_clock.cc:648] HybridClock initialized: now 1786127608768586 us; error 56 us; skew 500 ppm
I20260807 18:33:28.771166 16566 webserver.cc:460] Webserver started at http://127.11.14.66:36595/ using document root <none> and password file <none>
I20260807 18:33:28.772053 16566 fs_manager.cc:359] Metadata directory not provided
I20260807 18:33:28.772164 16566 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:33:28.772439 16566 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:33:28.774919 16566 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786127579900523-11321-0/minicluster-data/ts-1/data/instance:
uuid: "7fbf8f3479cd45b1b04df34ee4e200fe"
format_stamp: "Formatted at 2026-08-07 18:33:28 on dist-test-slave-6k22"
server_key: "f4c7a3d2c0e74124d614adf880a0720c"
server_key_iv: "af1c8371c8ca9a8696e2b83fcd87a912"
server_key_version: "encryptionkey@0"
I20260807 18:33:28.775499 16566 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786127579900523-11321-0/minicluster-data/ts-1/wal/instance:
uuid: "7fbf8f3479cd45b1b04df34ee4e200fe"
format_stamp: "Formatted at 2026-08-07 18:33:28 on dist-test-slave-6k22"
server_key: "f4c7a3d2c0e74124d614adf880a0720c"
server_key_iv: "af1c8371c8ca9a8696e2b83fcd87a912"
server_key_version: "encryptionkey@0"
I20260807 18:33:28.779362 16566 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260807 18:33:28.782126 16580 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:33:28.783614 16566 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 18:33:28.783802 16566 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786127579900523-11321-0/minicluster-data/ts-1/data,/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786127579900523-11321-0/minicluster-data/ts-1/wal
uuid: "7fbf8f3479cd45b1b04df34ee4e200fe"
format_stamp: "Formatted at 2026-08-07 18:33:28 on dist-test-slave-6k22"
server_key: "f4c7a3d2c0e74124d614adf880a0720c"
server_key_iv: "af1c8371c8ca9a8696e2b83fcd87a912"
server_key_version: "encryptionkey@0"
I20260807 18:33:28.783902 16566 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786127579900523-11321-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786127579900523-11321-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786127579900523-11321-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:33:28.789171 16566 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:33:28.789800 16566 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:33:28.789979 16566 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:33:28.790521 16566 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:33:28.791443 16566 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:33:28.791497 16566 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:33:28.791567 16566 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:33:28.791630 16566 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:33:28.800388 16566 rpc_server.cc:307] RPC server started. Bound to: 127.11.14.66:39587
I20260807 18:33:28.800422 16703 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.14.66:39587 every 8 connection(s)
I20260807 18:33:28.801291 16566 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786127579900523-11321-0/minicluster-data/ts-1/data/info.pb
I20260807 18:33:28.805177 11321 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkz56XN/build/debug/bin/kudu as pid 16566
I20260807 18:33:28.805298 11321 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786127579900523-11321-0/minicluster-data/ts-1/wal/instance
I20260807 18:33:28.805585 11321 external_mini_cluster.cc:1428] Setting key deed89f8eacd6b0efc3e87d2aa8a5826
I20260807 18:33:28.808127 11321 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkz56XN/build/debug/bin/kudu
/tmp/dist-test-taskkz56XN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786127579900523-11321-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786127579900523-11321-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786127579900523-11321-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786127579900523-11321-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.14.67:0
--local_ip_for_outbound_sockets=127.11.14.67
--webserver_interface=127.11.14.67
--webserver_port=0
--tserver_master_addrs=127.11.14.126:38827
--builtin_ntp_servers=127.11.14.84:35233
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260807 18:33:28.808430 16404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:28.803279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.66:49205 (local address 127.11.14.126:38827)
0807 18:33:28.803425 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:28.803428 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:28.804522 (+ 1094us) server_negotiation.cc:407] Connection header received
0807 18:33:28.806218 (+ 1696us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:28.806222 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:28.806265 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:28.806372 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:33:28.807870 (+ 1498us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:33:28.807873 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:33:28.807875 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:33:28.807887 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:33:28.807909 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:33:28.807917 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:33:28.807993 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:28.808196 (+ 203us) server_negotiation.cc:299] Negotiation successful
0807 18:33:28.808286 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260807 18:33:28.809629 16706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:28.803715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:38827 (local address 127.11.14.66:49205)
0807 18:33:28.804370 (+ 655us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:28.804411 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:28.805929 (+ 1518us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:28.806559 (+ 630us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:28.806568 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:28.806674 (+ 106us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:33:28.807420 (+ 746us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 18:33:28.807424 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:33:28.807483 (+ 59us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 18:33:28.807691 (+ 208us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:33:28.808018 (+ 327us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:33:28.808029 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:33:28.808035 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 18:33:28.808236 (+ 201us) client_negotiation.cc:240] Negotiation successful
0807 18:33:28.808504 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":117,"threads_started":1}
I20260807 18:33:28.810956 16704 heartbeater.cc:347] Connected to a master server at 127.11.14.126:38827
I20260807 18:33:28.811219 16704 heartbeater.cc:464] Registering TS with master...
I20260807 18:33:28.811810 16704 heartbeater.cc:511] Master 127.11.14.126:38827 requested a full tablet report, sending...
I20260807 18:33:28.812978 16360 ts_manager.cc:194] Registered new tserver with Master: 7fbf8f3479cd45b1b04df34ee4e200fe (127.11.14.66:39587)
W20260807 18:33:28.934095 16707 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:33:28.934441 16707 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:33:28.934540 16707 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:33:28.938732 16707 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:33:28.938814 16707 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:33:28.938902 16707 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.14.67
I20260807 18:33:28.943801 16707 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.14.84:35233
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786127579900523-11321-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786127579900523-11321-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.14.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786127579900523-11321-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.14.67
--webserver_port=0
--tserver_master_addrs=127.11.14.126:38827
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16707
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.14.67
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786127579900523-11321-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:33:28.944944 16707 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:33:28.947074 16707 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:33:28.958074 16715 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:33:28.958122 16713 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:33:28.958137 16707 server_base.cc:1034] running on GCE node
W20260807 18:33:28.958293 16712 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:33:28.958766 16707 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:33:28.959429 16707 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:33:28.960623 16707 hybrid_clock.cc:648] HybridClock initialized: now 1786127608960592 us; error 43 us; skew 500 ppm
I20260807 18:33:28.962607 16707 webserver.cc:460] Webserver started at http://127.11.14.67:43847/ using document root <none> and password file <none>
I20260807 18:33:28.963223 16707 fs_manager.cc:359] Metadata directory not provided
I20260807 18:33:28.963292 16707 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:33:28.963464 16707 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:33:28.965637 16707 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786127579900523-11321-0/minicluster-data/ts-2/data/instance:
uuid: "a03fc146d7c041e29c3a4b8609a32e3e"
format_stamp: "Formatted at 2026-08-07 18:33:28 on dist-test-slave-6k22"
server_key: "b3158c43a9d74fbcd0dfc267f67fd7bd"
server_key_iv: "c7af62b02b1c0ab110a637cd123473f7"
server_key_version: "encryptionkey@0"
I20260807 18:33:28.966240 16707 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786127579900523-11321-0/minicluster-data/ts-2/wal/instance:
uuid: "a03fc146d7c041e29c3a4b8609a32e3e"
format_stamp: "Formatted at 2026-08-07 18:33:28 on dist-test-slave-6k22"
server_key: "b3158c43a9d74fbcd0dfc267f67fd7bd"
server_key_iv: "c7af62b02b1c0ab110a637cd123473f7"
server_key_version: "encryptionkey@0"
I20260807 18:33:28.970235 16707 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 18:33:28.973033 16721 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:33:28.974397 16707 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 18:33:28.974538 16707 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786127579900523-11321-0/minicluster-data/ts-2/data,/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786127579900523-11321-0/minicluster-data/ts-2/wal
uuid: "a03fc146d7c041e29c3a4b8609a32e3e"
format_stamp: "Formatted at 2026-08-07 18:33:28 on dist-test-slave-6k22"
server_key: "b3158c43a9d74fbcd0dfc267f67fd7bd"
server_key_iv: "c7af62b02b1c0ab110a637cd123473f7"
server_key_version: "encryptionkey@0"
I20260807 18:33:28.974650 16707 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786127579900523-11321-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786127579900523-11321-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786127579900523-11321-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:33:28.979386 16707 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:33:28.980054 16707 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:33:28.980247 16707 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:33:28.980820 16707 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:33:28.981753 16707 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:33:28.981801 16707 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:33:28.981866 16707 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:33:28.981899 16707 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:33:28.991961 16707 rpc_server.cc:307] RPC server started. Bound to: 127.11.14.67:35671
I20260807 18:33:28.992040 16844 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.14.67:35671 every 8 connection(s)
I20260807 18:33:28.992944 16707 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786127579900523-11321-0/minicluster-data/ts-2/data/info.pb
I20260807 18:33:28.996658 11321 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkz56XN/build/debug/bin/kudu as pid 16707
I20260807 18:33:28.996829 11321 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786127579900523-11321-0/minicluster-data/ts-2/wal/instance
I20260807 18:33:28.997123 11321 external_mini_cluster.cc:1428] Setting key 993fa66983fd6596faf5e84ddc55fd97
I20260807 18:33:28.999936 16404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:28.994962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.67:41283 (local address 127.11.14.126:38827)
0807 18:33:28.995108 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:28.995112 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:28.996200 (+ 1088us) server_negotiation.cc:407] Connection header received
0807 18:33:28.997824 (+ 1624us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:28.997828 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:28.997880 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:28.997959 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:33:28.999411 (+ 1452us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:33:28.999414 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:33:28.999416 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:33:28.999428 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:33:28.999448 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:33:28.999456 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:33:28.999529 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:28.999746 (+ 217us) server_negotiation.cc:299] Negotiation successful
0807 18:33:28.999788 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260807 18:33:29.000869 16847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:28.995297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:38827 (local address 127.11.14.67:41283)
0807 18:33:28.996044 (+ 747us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:28.996086 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:28.997592 (+ 1506us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:28.998200 (+ 608us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:28.998210 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:28.998323 (+ 113us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:33:28.999084 (+ 761us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 18:33:28.999086 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:33:28.999157 (+ 71us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 18:33:28.999250 (+ 93us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:33:28.999548 (+ 298us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:33:28.999580 (+ 32us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:33:28.999586 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 18:33:28.999788 (+ 202us) client_negotiation.cc:240] Negotiation successful
0807 18:33:29.000071 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":320,"thread_start_us":117,"threads_started":1}
I20260807 18:33:29.002054 16845 heartbeater.cc:347] Connected to a master server at 127.11.14.126:38827
I20260807 18:33:29.002314 16845 heartbeater.cc:464] Registering TS with master...
I20260807 18:33:29.002750 16845 heartbeater.cc:511] Master 127.11.14.126:38827 requested a full tablet report, sending...
I20260807 18:33:29.003904 16360 ts_manager.cc:194] Registered new tserver with Master: a03fc146d7c041e29c3a4b8609a32e3e (127.11.14.67:35671)
I20260807 18:33:29.012074 11321 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 18:33:29.016031 16404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:29.014159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34882 (local address 127.11.14.126:38827)
0807 18:33:29.014297 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:29.014302 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:29.014487 (+ 185us) server_negotiation.cc:407] Connection header received
0807 18:33:29.014659 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:29.014662 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:29.014708 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:29.014782 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:33:29.015615 (+ 833us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:33:29.015619 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:33:29.015621 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:33:29.015633 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:33:29.015654 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:33:29.015669 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:33:29.015763 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:29.015838 (+ 75us) server_negotiation.cc:299] Negotiation successful
0807 18:33:29.015879 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260807 18:33:29.019166 16360 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34882:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 18:33:29.022324 16360 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 18:33:29.035684 16859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:29.031836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35626 (local address 127.11.14.66:39587)
0807 18:33:29.032415 (+ 579us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:29.032428 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:29.032856 (+ 428us) server_negotiation.cc:407] Connection header received
0807 18:33:29.033089 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:29.033096 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:29.033324 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:29.033442 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:33:29.034917 (+ 1475us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:33:29.034924 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:33:29.034937 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:33:29.034949 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:33:29.035001 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:33:29.035043 (+ 42us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:33:29.035159 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:29.035394 (+ 235us) server_negotiation.cc:299] Negotiation successful
0807 18:33:29.035481 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":449,"thread_start_us":146,"threads_started":1}
I20260807 18:33:29.035871 16862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:29.032446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50846 (local address 127.11.14.65:37117)
0807 18:33:29.033081 (+ 635us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:29.033093 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:29.033116 (+ 23us) server_negotiation.cc:407] Connection header received
0807 18:33:29.033200 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:29.033209 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:29.033408 (+ 199us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:29.033567 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:33:29.035114 (+ 1547us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:33:29.035119 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:33:29.035129 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:33:29.035141 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:33:29.035194 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:33:29.035223 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:33:29.035330 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:29.035640 (+ 310us) server_negotiation.cc:299] Negotiation successful
0807 18:33:29.035714 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":522,"thread_start_us":152,"threads_started":1}
I20260807 18:33:29.035941 16858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:29.031441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.67:35671 (local address 127.0.0.1:40406)
0807 18:33:29.032879 (+ 1438us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:29.032911 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:29.033266 (+ 355us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:29.034190 (+ 924us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:29.034193 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:29.034208 (+ 15us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:33:29.034777 (+ 569us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 18:33:29.034780 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:33:29.034791 (+ 11us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 18:33:29.034816 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:33:29.035249 (+ 433us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:33:29.035252 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:33:29.035255 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 18:33:29.035340 (+ 85us) client_negotiation.cc:240] Negotiation successful
0807 18:33:29.035412 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":697,"thread_start_us":121,"threads_started":1}
I20260807 18:33:29.036092 16856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:29.032276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.65:37117 (local address 127.0.0.1:50846)
0807 18:33:29.032847 (+ 571us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:29.032860 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:29.032957 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:29.033568 (+ 611us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:29.033575 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:29.033666 (+ 91us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:33:29.034286 (+ 620us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 18:33:29.034288 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:33:29.034310 (+ 22us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 18:33:29.034404 (+ 94us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:33:29.035371 (+ 967us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:33:29.035374 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:33:29.035377 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 18:33:29.035445 (+ 68us) client_negotiation.cc:240] Negotiation successful
0807 18:33:29.035479 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":501,"spinlock_wait_cycles":5376,"thread_start_us":97,"threads_started":1}
I20260807 18:33:29.036093 16861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:29.031835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.66:39587 (local address 127.0.0.1:35626)
0807 18:33:29.032705 (+ 870us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:29.032763 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:29.032957 (+ 194us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:29.034190 (+ 1233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:29.034193 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:29.034208 (+ 15us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:33:29.034690 (+ 482us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 18:33:29.034692 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:33:29.034704 (+ 12us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 18:33:29.034758 (+ 54us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:33:29.035210 (+ 452us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:33:29.035221 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:33:29.035227 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 18:33:29.035864 (+ 637us) client_negotiation.cc:240] Negotiation successful
0807 18:33:29.035904 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":654,"spinlock_wait_cycles":7680,"thread_start_us":96,"threads_started":1}
I20260807 18:33:29.036329 16857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:29.031665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40406 (local address 127.11.14.67:35671)
0807 18:33:29.032704 (+ 1039us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:29.032717 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:29.033285 (+ 568us) server_negotiation.cc:407] Connection header received
0807 18:33:29.033350 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:29.033356 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:29.033509 (+ 153us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:29.033641 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:33:29.034942 (+ 1301us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:33:29.034948 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:33:29.034962 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:33:29.034979 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:33:29.035050 (+ 71us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:33:29.035091 (+ 41us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:33:29.035206 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:29.035587 (+ 381us) server_negotiation.cc:299] Negotiation successful
0807 18:33:29.035664 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":921,"thread_start_us":144,"threads_started":1}
I20260807 18:33:29.039194 16630 tablet_service.cc:1467] Processing CreateTablet for tablet 80e561f7af2c4985ba330fb41bf474d6 (DEFAULT_TABLE table=test-table [id=06fbbc719c2841939c8b06cb4beed537]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:33:29.039206 16772 tablet_service.cc:1467] Processing CreateTablet for tablet 80e561f7af2c4985ba330fb41bf474d6 (DEFAULT_TABLE table=test-table [id=06fbbc719c2841939c8b06cb4beed537]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:33:29.039206 16488 tablet_service.cc:1467] Processing CreateTablet for tablet 80e561f7af2c4985ba330fb41bf474d6 (DEFAULT_TABLE table=test-table [id=06fbbc719c2841939c8b06cb4beed537]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:33:29.040279 16630 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 80e561f7af2c4985ba330fb41bf474d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:33:29.040313 16772 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 80e561f7af2c4985ba330fb41bf474d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:33:29.040313 16488 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 80e561f7af2c4985ba330fb41bf474d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:33:29.047781 16865 tablet_bootstrap.cc:492] T 80e561f7af2c4985ba330fb41bf474d6 P 28b3d41c8dbf482e828645377eac64a0: Bootstrap starting.
I20260807 18:33:29.046658 16863 tablet_bootstrap.cc:492] T 80e561f7af2c4985ba330fb41bf474d6 P 7fbf8f3479cd45b1b04df34ee4e200fe: Bootstrap starting.
I20260807 18:33:29.048408 16864 tablet_bootstrap.cc:492] T 80e561f7af2c4985ba330fb41bf474d6 P a03fc146d7c041e29c3a4b8609a32e3e: Bootstrap starting.
I20260807 18:33:29.050205 16864 tablet_bootstrap.cc:654] T 80e561f7af2c4985ba330fb41bf474d6 P a03fc146d7c041e29c3a4b8609a32e3e: Neither blocks nor log segments found. Creating new log.
I20260807 18:33:29.050230 16865 tablet_bootstrap.cc:654] T 80e561f7af2c4985ba330fb41bf474d6 P 28b3d41c8dbf482e828645377eac64a0: Neither blocks nor log segments found. Creating new log.
I20260807 18:33:29.050230 16863 tablet_bootstrap.cc:654] T 80e561f7af2c4985ba330fb41bf474d6 P 7fbf8f3479cd45b1b04df34ee4e200fe: Neither blocks nor log segments found. Creating new log.
I20260807 18:33:29.051012 16864 log.cc:824] T 80e561f7af2c4985ba330fb41bf474d6 P a03fc146d7c041e29c3a4b8609a32e3e: Log is configured to *not* fsync() on all Append() calls
I20260807 18:33:29.051115 16865 log.cc:824] T 80e561f7af2c4985ba330fb41bf474d6 P 28b3d41c8dbf482e828645377eac64a0: Log is configured to *not* fsync() on all Append() calls
I20260807 18:33:29.051115 16863 log.cc:824] T 80e561f7af2c4985ba330fb41bf474d6 P 7fbf8f3479cd45b1b04df34ee4e200fe: Log is configured to *not* fsync() on all Append() calls
I20260807 18:33:29.053155 16865 tablet_bootstrap.cc:492] T 80e561f7af2c4985ba330fb41bf474d6 P 28b3d41c8dbf482e828645377eac64a0: No bootstrap required, opened a new log
I20260807 18:33:29.053289 16863 tablet_bootstrap.cc:492] T 80e561f7af2c4985ba330fb41bf474d6 P 7fbf8f3479cd45b1b04df34ee4e200fe: No bootstrap required, opened a new log
I20260807 18:33:29.053341 16864 tablet_bootstrap.cc:492] T 80e561f7af2c4985ba330fb41bf474d6 P a03fc146d7c041e29c3a4b8609a32e3e: No bootstrap required, opened a new log
I20260807 18:33:29.053370 16865 ts_tablet_manager.cc:1397] T 80e561f7af2c4985ba330fb41bf474d6 P 28b3d41c8dbf482e828645377eac64a0: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260807 18:33:29.053539 16864 ts_tablet_manager.cc:1397] T 80e561f7af2c4985ba330fb41bf474d6 P a03fc146d7c041e29c3a4b8609a32e3e: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260807 18:33:29.053663 16863 ts_tablet_manager.cc:1397] T 80e561f7af2c4985ba330fb41bf474d6 P 7fbf8f3479cd45b1b04df34ee4e200fe: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260807 18:33:29.057019 16865 raft_consensus.cc:348] T 80e561f7af2c4985ba330fb41bf474d6 P 28b3d41c8dbf482e828645377eac64a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a03fc146d7c041e29c3a4b8609a32e3e" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 35671 } } peers { permanent_uuid: "7fbf8f3479cd45b1b04df34ee4e200fe" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 39587 } } peers { permanent_uuid: "28b3d41c8dbf482e828645377eac64a0" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 37117 } }
I20260807 18:33:29.057274 16865 raft_consensus.cc:374] T 80e561f7af2c4985ba330fb41bf474d6 P 28b3d41c8dbf482e828645377eac64a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:33:29.057329 16865 raft_consensus.cc:729] T 80e561f7af2c4985ba330fb41bf474d6 P 28b3d41c8dbf482e828645377eac64a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28b3d41c8dbf482e828645377eac64a0, State: Initialized, Role: FOLLOWER
I20260807 18:33:29.057786 16865 consensus_queue.cc:260] T 80e561f7af2c4985ba330fb41bf474d6 P 28b3d41c8dbf482e828645377eac64a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a03fc146d7c041e29c3a4b8609a32e3e" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 35671 } } peers { permanent_uuid: "7fbf8f3479cd45b1b04df34ee4e200fe" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 39587 } } peers { permanent_uuid: "28b3d41c8dbf482e828645377eac64a0" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 37117 } }
I20260807 18:33:29.058139 16864 raft_consensus.cc:348] T 80e561f7af2c4985ba330fb41bf474d6 P a03fc146d7c041e29c3a4b8609a32e3e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a03fc146d7c041e29c3a4b8609a32e3e" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 35671 } } peers { permanent_uuid: "7fbf8f3479cd45b1b04df34ee4e200fe" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 39587 } } peers { permanent_uuid: "28b3d41c8dbf482e828645377eac64a0" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 37117 } }
I20260807 18:33:29.058139 16863 raft_consensus.cc:348] T 80e561f7af2c4985ba330fb41bf474d6 P 7fbf8f3479cd45b1b04df34ee4e200fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a03fc146d7c041e29c3a4b8609a32e3e" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 35671 } } peers { permanent_uuid: "7fbf8f3479cd45b1b04df34ee4e200fe" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 39587 } } peers { permanent_uuid: "28b3d41c8dbf482e828645377eac64a0" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 37117 } }
I20260807 18:33:29.058418 16863 raft_consensus.cc:374] T 80e561f7af2c4985ba330fb41bf474d6 P 7fbf8f3479cd45b1b04df34ee4e200fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:33:29.058418 16864 raft_consensus.cc:374] T 80e561f7af2c4985ba330fb41bf474d6 P a03fc146d7c041e29c3a4b8609a32e3e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:33:29.058471 16863 raft_consensus.cc:729] T 80e561f7af2c4985ba330fb41bf474d6 P 7fbf8f3479cd45b1b04df34ee4e200fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7fbf8f3479cd45b1b04df34ee4e200fe, State: Initialized, Role: FOLLOWER
I20260807 18:33:29.058480 16864 raft_consensus.cc:729] T 80e561f7af2c4985ba330fb41bf474d6 P a03fc146d7c041e29c3a4b8609a32e3e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a03fc146d7c041e29c3a4b8609a32e3e, State: Initialized, Role: FOLLOWER
I20260807 18:33:29.058483 16563 heartbeater.cc:503] Master 127.11.14.126:38827 was elected leader, sending a full tablet report...
I20260807 18:33:29.058969 16865 ts_tablet_manager.cc:1428] T 80e561f7af2c4985ba330fb41bf474d6 P 28b3d41c8dbf482e828645377eac64a0: Time spent starting tablet: real 0.006s user 0.001s sys 0.005s
I20260807 18:33:29.058928 16863 consensus_queue.cc:260] T 80e561f7af2c4985ba330fb41bf474d6 P 7fbf8f3479cd45b1b04df34ee4e200fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a03fc146d7c041e29c3a4b8609a32e3e" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 35671 } } peers { permanent_uuid: "7fbf8f3479cd45b1b04df34ee4e200fe" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 39587 } } peers { permanent_uuid: "28b3d41c8dbf482e828645377eac64a0" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 37117 } }
I20260807 18:33:29.058928 16864 consensus_queue.cc:260] T 80e561f7af2c4985ba330fb41bf474d6 P a03fc146d7c041e29c3a4b8609a32e3e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a03fc146d7c041e29c3a4b8609a32e3e" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 35671 } } peers { permanent_uuid: "7fbf8f3479cd45b1b04df34ee4e200fe" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 39587 } } peers { permanent_uuid: "28b3d41c8dbf482e828645377eac64a0" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 37117 } }
I20260807 18:33:29.059842 16845 heartbeater.cc:503] Master 127.11.14.126:38827 was elected leader, sending a full tablet report...
I20260807 18:33:29.059919 16864 ts_tablet_manager.cc:1428] T 80e561f7af2c4985ba330fb41bf474d6 P a03fc146d7c041e29c3a4b8609a32e3e: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260807 18:33:29.059919 16863 ts_tablet_manager.cc:1428] T 80e561f7af2c4985ba330fb41bf474d6 P 7fbf8f3479cd45b1b04df34ee4e200fe: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260807 18:33:29.060242 16704 heartbeater.cc:503] Master 127.11.14.126:38827 was elected leader, sending a full tablet report...
W20260807 18:33:29.122848 16564 tablet.cc:2369] T 80e561f7af2c4985ba330fb41bf474d6 P 28b3d41c8dbf482e828645377eac64a0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:33:29.186407 16872 raft_consensus.cc:482] T 80e561f7af2c4985ba330fb41bf474d6 P 7fbf8f3479cd45b1b04df34ee4e200fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:33:29.186698 16872 raft_consensus.cc:504] T 80e561f7af2c4985ba330fb41bf474d6 P 7fbf8f3479cd45b1b04df34ee4e200fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a03fc146d7c041e29c3a4b8609a32e3e" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 35671 } } peers { permanent_uuid: "7fbf8f3479cd45b1b04df34ee4e200fe" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 39587 } } peers { permanent_uuid: "28b3d41c8dbf482e828645377eac64a0" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 37117 } }
I20260807 18:33:29.188133 16872 leader_election.cc:288] T 80e561f7af2c4985ba330fb41bf474d6 P 7fbf8f3479cd45b1b04df34ee4e200fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a03fc146d7c041e29c3a4b8609a32e3e (127.11.14.67:35671), 28b3d41c8dbf482e828645377eac64a0 (127.11.14.65:37117)
I20260807 18:33:29.190147 16706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:29.188372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.67:35671 (local address 127.11.14.66:33461)
0807 18:33:29.188544 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:29.188562 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:29.188655 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:29.188936 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:29.188940 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:29.188956 (+ 16us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:33:29.189535 (+ 579us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 18:33:29.189540 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:33:29.189554 (+ 14us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 18:33:29.189586 (+ 32us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:33:29.189895 (+ 309us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:33:29.189901 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:33:29.189908 (+ 7us) client_negotiation.cc:769] Sending connection context
0807 18:33:29.189982 (+ 74us) client_negotiation.cc:240] Negotiation successful
0807 18:33:29.190024 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":79}
I20260807 18:33:29.190198 16857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:29.188514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.66:33461 (local address 127.11.14.67:35671)
0807 18:33:29.188641 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:29.188646 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:29.188665 (+ 19us) server_negotiation.cc:407] Connection header received
0807 18:33:29.188753 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:29.188759 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:29.188802 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:29.188895 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:33:29.189726 (+ 831us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:33:29.189731 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:33:29.189735 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:33:29.189748 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:33:29.189771 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:33:29.189785 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:33:29.189901 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:29.190011 (+ 110us) server_negotiation.cc:299] Negotiation successful
0807 18:33:29.190053 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260807 18:33:29.190488 16862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:29.188564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.66:58857 (local address 127.11.14.65:37117)
0807 18:33:29.188710 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:29.188717 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:29.189097 (+ 380us) server_negotiation.cc:407] Connection header received
0807 18:33:29.189198 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:29.189203 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:29.189241 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:29.189328 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:33:29.190008 (+ 680us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:33:29.190012 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:33:29.190014 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:33:29.190030 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:33:29.190052 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:33:29.190062 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:33:29.190149 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:29.190323 (+ 174us) server_negotiation.cc:299] Negotiation successful
0807 18:33:29.190373 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260807 18:33:29.190837 16873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:29.188372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.65:37117 (local address 127.11.14.66:58857)
0807 18:33:29.188985 (+ 613us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:29.189000 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:29.189096 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:29.189352 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:29.189374 (+ 22us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:29.189393 (+ 19us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:33:29.189819 (+ 426us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 18:33:29.189820 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:33:29.189828 (+ 8us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 18:33:29.189867 (+ 39us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:33:29.190172 (+ 305us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:33:29.190176 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:33:29.190180 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 18:33:29.190646 (+ 466us) client_negotiation.cc:240] Negotiation successful
0807 18:33:29.190688 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":549,"mutex_wait_us":197,"thread_start_us":156,"threads_started":1}
I20260807 18:33:29.190786 16794 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80e561f7af2c4985ba330fb41bf474d6" candidate_uuid: "7fbf8f3479cd45b1b04df34ee4e200fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a03fc146d7c041e29c3a4b8609a32e3e" is_pre_election: true
I20260807 18:33:29.191111 16794 raft_consensus.cc:2454] T 80e561f7af2c4985ba330fb41bf474d6 P a03fc146d7c041e29c3a4b8609a32e3e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fbf8f3479cd45b1b04df34ee4e200fe in term 0.
I20260807 18:33:29.191493 16511 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80e561f7af2c4985ba330fb41bf474d6" candidate_uuid: "7fbf8f3479cd45b1b04df34ee4e200fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28b3d41c8dbf482e828645377eac64a0" is_pre_election: true
I20260807 18:33:29.191625 16581 leader_election.cc:302] T 80e561f7af2c4985ba330fb41bf474d6 P 7fbf8f3479cd45b1b04df34ee4e200fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fbf8f3479cd45b1b04df34ee4e200fe, a03fc146d7c041e29c3a4b8609a32e3e; no voters:
I20260807 18:33:29.191812 16511 raft_consensus.cc:2454] T 80e561f7af2c4985ba330fb41bf474d6 P 28b3d41c8dbf482e828645377eac64a0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fbf8f3479cd45b1b04df34ee4e200fe in term 0.
I20260807 18:33:29.191994 16872 raft_consensus.cc:2781] T 80e561f7af2c4985ba330fb41bf474d6 P 7fbf8f3479cd45b1b04df34ee4e200fe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:33:29.192126 16872 raft_consensus.cc:482] T 80e561f7af2c4985ba330fb41bf474d6 P 7fbf8f3479cd45b1b04df34ee4e200fe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:33:29.192193 16872 raft_consensus.cc:3037] T 80e561f7af2c4985ba330fb41bf474d6 P 7fbf8f3479cd45b1b04df34ee4e200fe [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:33:29.193320 16872 raft_consensus.cc:504] T 80e561f7af2c4985ba330fb41bf474d6 P 7fbf8f3479cd45b1b04df34ee4e200fe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a03fc146d7c041e29c3a4b8609a32e3e" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 35671 } } peers { permanent_uuid: "7fbf8f3479cd45b1b04df34ee4e200fe" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 39587 } } peers { permanent_uuid: "28b3d41c8dbf482e828645377eac64a0" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 37117 } }
I20260807 18:33:29.193665 16872 leader_election.cc:288] T 80e561f7af2c4985ba330fb41bf474d6 P 7fbf8f3479cd45b1b04df34ee4e200fe [CANDIDATE]: Term 1 election: Requested vote from peers a03fc146d7c041e29c3a4b8609a32e3e (127.11.14.67:35671), 28b3d41c8dbf482e828645377eac64a0 (127.11.14.65:37117)
I20260807 18:33:29.194049 16794 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80e561f7af2c4985ba330fb41bf474d6" candidate_uuid: "7fbf8f3479cd45b1b04df34ee4e200fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a03fc146d7c041e29c3a4b8609a32e3e"
I20260807 18:33:29.194141 16511 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80e561f7af2c4985ba330fb41bf474d6" candidate_uuid: "7fbf8f3479cd45b1b04df34ee4e200fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28b3d41c8dbf482e828645377eac64a0"
I20260807 18:33:29.194216 16794 raft_consensus.cc:3037] T 80e561f7af2c4985ba330fb41bf474d6 P a03fc146d7c041e29c3a4b8609a32e3e [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:33:29.194278 16511 raft_consensus.cc:3037] T 80e561f7af2c4985ba330fb41bf474d6 P 28b3d41c8dbf482e828645377eac64a0 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:33:29.195322 16511 raft_consensus.cc:2454] T 80e561f7af2c4985ba330fb41bf474d6 P 28b3d41c8dbf482e828645377eac64a0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fbf8f3479cd45b1b04df34ee4e200fe in term 1.
I20260807 18:33:29.195325 16794 raft_consensus.cc:2454] T 80e561f7af2c4985ba330fb41bf474d6 P a03fc146d7c041e29c3a4b8609a32e3e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fbf8f3479cd45b1b04df34ee4e200fe in term 1.
I20260807 18:33:29.195627 16583 leader_election.cc:302] T 80e561f7af2c4985ba330fb41bf474d6 P 7fbf8f3479cd45b1b04df34ee4e200fe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28b3d41c8dbf482e828645377eac64a0, 7fbf8f3479cd45b1b04df34ee4e200fe; no voters:
I20260807 18:33:29.195819 16872 raft_consensus.cc:2781] T 80e561f7af2c4985ba330fb41bf474d6 P 7fbf8f3479cd45b1b04df34ee4e200fe [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:33:29.196062 16872 raft_consensus.cc:686] T 80e561f7af2c4985ba330fb41bf474d6 P 7fbf8f3479cd45b1b04df34ee4e200fe [term 1 LEADER]: Becoming Leader. State: Replica: 7fbf8f3479cd45b1b04df34ee4e200fe, State: Running, Role: LEADER
I20260807 18:33:29.196409 16872 consensus_queue.cc:237] T 80e561f7af2c4985ba330fb41bf474d6 P 7fbf8f3479cd45b1b04df34ee4e200fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a03fc146d7c041e29c3a4b8609a32e3e" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 35671 } } peers { permanent_uuid: "7fbf8f3479cd45b1b04df34ee4e200fe" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 39587 } } peers { permanent_uuid: "28b3d41c8dbf482e828645377eac64a0" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 37117 } }
I20260807 18:33:29.200048 16357 catalog_manager.cc:5495] T 80e561f7af2c4985ba330fb41bf474d6 P 7fbf8f3479cd45b1b04df34ee4e200fe reported cstate change: term changed from 0 to 1, leader changed from <none> to 7fbf8f3479cd45b1b04df34ee4e200fe (127.11.14.66). New cstate: current_term: 1 leader_uuid: "7fbf8f3479cd45b1b04df34ee4e200fe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a03fc146d7c041e29c3a4b8609a32e3e" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 35671 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7fbf8f3479cd45b1b04df34ee4e200fe" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 39587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "28b3d41c8dbf482e828645377eac64a0" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 37117 } health_report { overall_health: UNKNOWN } } }
I20260807 18:33:29.226343 16859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:29.224693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35636 (local address 127.11.14.66:39587)
0807 18:33:29.224831 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:29.224835 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:29.224849 (+ 14us) server_negotiation.cc:407] Connection header received
0807 18:33:29.224932 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:29.224935 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:29.224997 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:29.225073 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:33:29.225972 (+ 899us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:33:29.225975 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:33:29.225977 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:33:29.225989 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:33:29.226011 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:33:29.226020 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:33:29.226113 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:29.226186 (+ 73us) server_negotiation.cc:299] Negotiation successful
0807 18:33:29.226222 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260807 18:33:29.232662 16794 raft_consensus.cc:1261] T 80e561f7af2c4985ba330fb41bf474d6 P a03fc146d7c041e29c3a4b8609a32e3e [term 1 FOLLOWER]: Refusing update from remote peer 7fbf8f3479cd45b1b04df34ee4e200fe: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:33:29.232662 16511 raft_consensus.cc:1261] T 80e561f7af2c4985ba330fb41bf474d6 P 28b3d41c8dbf482e828645377eac64a0 [term 1 FOLLOWER]: Refusing update from remote peer 7fbf8f3479cd45b1b04df34ee4e200fe: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:33:29.233440 16872 consensus_queue.cc:1035] T 80e561f7af2c4985ba330fb41bf474d6 P 7fbf8f3479cd45b1b04df34ee4e200fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "28b3d41c8dbf482e828645377eac64a0" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 37117 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:33:29.233666 16874 consensus_queue.cc:1035] T 80e561f7af2c4985ba330fb41bf474d6 P 7fbf8f3479cd45b1b04df34ee4e200fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "a03fc146d7c041e29c3a4b8609a32e3e" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 35671 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:33:29.241811 16877 mvcc.cc:205] Tried to move back new op lower bound from 7315978687409156096 to 7315978687271284736. Current Snapshot: MvccSnapshot[applied={T|T < 7315978687409156096}]
I20260807 18:33:29.245165 16880 mvcc.cc:205] Tried to move back new op lower bound from 7315978687409156096 to 7315978687271284736. Current Snapshot: MvccSnapshot[applied={T|T < 7315978687409156096}]
I20260807 18:33:29.250793 16359 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:34882:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:33:29.250957 16359 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:34882:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:33:29.253666 16359 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fab2b364e6c0404f961b778da8c21f86: Sending DeleteTablet for 3 replicas of tablet 80e561f7af2c4985ba330fb41bf474d6
I20260807 18:33:29.254448 16772 tablet_service.cc:1514] Processing DeleteTablet for tablet 80e561f7af2c4985ba330fb41bf474d6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:33:29 UTC) from {username='slave'} at 127.0.0.1:40406
I20260807 18:33:29.254631 16630 tablet_service.cc:1514] Processing DeleteTablet for tablet 80e561f7af2c4985ba330fb41bf474d6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:33:29 UTC) from {username='slave'} at 127.0.0.1:35626
I20260807 18:33:29.254635 16488 tablet_service.cc:1514] Processing DeleteTablet for tablet 80e561f7af2c4985ba330fb41bf474d6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:33:29 UTC) from {username='slave'} at 127.0.0.1:50846
I20260807 18:33:29.255117 16888 tablet_replica.cc:330] T 80e561f7af2c4985ba330fb41bf474d6 P a03fc146d7c041e29c3a4b8609a32e3e: stopping tablet replica
I20260807 18:33:29.255220 16889 tablet_replica.cc:330] T 80e561f7af2c4985ba330fb41bf474d6 P 7fbf8f3479cd45b1b04df34ee4e200fe: stopping tablet replica
I20260807 18:33:29.255580 16888 raft_consensus.cc:2229] T 80e561f7af2c4985ba330fb41bf474d6 P a03fc146d7c041e29c3a4b8609a32e3e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:33:29.255602 16889 raft_consensus.cc:2229] T 80e561f7af2c4985ba330fb41bf474d6 P 7fbf8f3479cd45b1b04df34ee4e200fe [term 1 LEADER]: Raft consensus shutting down.
I20260807 18:33:29.255990 16888 raft_consensus.cc:2258] T 80e561f7af2c4985ba330fb41bf474d6 P a03fc146d7c041e29c3a4b8609a32e3e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:33:29.256006 11321 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkz56XN/build/debug/bin/kudu with pid 16425
I20260807 18:33:29.256014 16889 raft_consensus.cc:2258] T 80e561f7af2c4985ba330fb41bf474d6 P 7fbf8f3479cd45b1b04df34ee4e200fe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:33:29.262240 16888 ts_tablet_manager.cc:1905] T 80e561f7af2c4985ba330fb41bf474d6 P a03fc146d7c041e29c3a4b8609a32e3e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:33:29.265116 16888 ts_tablet_manager.cc:1918] T 80e561f7af2c4985ba330fb41bf474d6 P a03fc146d7c041e29c3a4b8609a32e3e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:33:29.265223 16888 log.cc:1196] T 80e561f7af2c4985ba330fb41bf474d6 P a03fc146d7c041e29c3a4b8609a32e3e: Deleting WAL directory at /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786127579900523-11321-0/minicluster-data/ts-2/wal/wals/80e561f7af2c4985ba330fb41bf474d6
I20260807 18:33:29.265561 16888 ts_tablet_manager.cc:1939] T 80e561f7af2c4985ba330fb41bf474d6 P a03fc146d7c041e29c3a4b8609a32e3e: Deleting consensus metadata
I20260807 18:33:29.266968 16342 catalog_manager.cc:4841] TS a03fc146d7c041e29c3a4b8609a32e3e (127.11.14.67:35671): tablet 80e561f7af2c4985ba330fb41bf474d6 (table test-table [id=06fbbc719c2841939c8b06cb4beed537]) successfully deleted
W20260807 18:33:29.271622 16345 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.14.65:37117 (error 108)
I20260807 18:33:29.272394 11321 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkz56XN/build/debug/bin/kudu with pid 16566
I20260807 18:33:29.273442 16858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:29.272606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.65:37117 (local address 127.0.0.1:50852)
0807 18:33:29.273289 (+ 683us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:29.273361 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.65:37117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":589}
W20260807 18:33:29.274683 16345 catalog_manager.cc:4584] TS 28b3d41c8dbf482e828645377eac64a0 (127.11.14.65:37117): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 80e561f7af2c4985ba330fb41bf474d6: Network error: Client connection negotiation failed: client connection to 127.11.14.65:37117: connect: Connection refused (error 111)
I20260807 18:33:29.280670 11321 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkz56XN/build/debug/bin/kudu with pid 16707
I20260807 18:33:29.283725 16858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:29.280744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.66:39587 (local address 127.0.0.1:35650)
0807 18:33:29.283396 (+ 2652us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:29.283467 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.66:39587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2523}
I20260807 18:33:29.288800 11321 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkz56XN/build/debug/bin/kudu with pid 16327
2026-08-07T18:33:29Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (1128 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260807 18:33:29.313799 11321 mini_oidc.cc:152] Starting JWKS server
I20260807 18:33:29.315627 11321 webserver.cc:460] Webserver started at https://127.0.0.1:33757/ using document root <none> and password file <none>
I20260807 18:33:29.315838 11321 mini_oidc.cc:189] Starting OIDC Discovery server
I20260807 18:33:29.316155 11321 webserver.cc:460] Webserver started at http://127.0.0.1:44087/ using document root <none> and password file <none>
2026-08-07T18:33:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:33:29Z Disabled control of system clock
I20260807 18:33:29.326804 11321 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkz56XN/build/debug/bin/kudu
/tmp/dist-test-taskkz56XN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786127579900523-11321-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786127579900523-11321-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786127579900523-11321-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786127579900523-11321-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.14.126:34369
--webserver_interface=127.11.14.126
--webserver_port=0
--builtin_ntp_servers=127.11.14.84:36929
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.14.126:34369
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33757/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786127579900523-11321-0/testchainca.cert with env {}
W20260807 18:33:29.442085 16897 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:33:29.442365 16897 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:33:29.442407 16897 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:33:29.446427 16897 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:33:29.446494 16897 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:33:29.446511 16897 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:33:29.446527 16897 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:33:29.446542 16897 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260807 18:33:29.446558 16897 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260807 18:33:29.446571 16897 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:33757/jwks.json
I20260807 18:33:29.452020 16897 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.14.84:36929
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786127579900523-11321-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786127579900523-11321-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.14.126:34369
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.14.126:34369
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33757/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786127579900523-11321-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.14.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786127579900523-11321-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16897
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786127579900523-11321-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:33:29.453143 16897 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:33:29.454164 16897 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:33:29.462908 16905 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:33:29.462915 16909 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:33:29.463021 16897 server_base.cc:1034] running on GCE node
W20260807 18:33:29.462914 16906 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:33:29.463822 16897 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:33:29.464969 16897 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:33:29.466140 16897 hybrid_clock.cc:648] HybridClock initialized: now 1786127609466106 us; error 46 us; skew 500 ppm
I20260807 18:33:29.468156 16897 webserver.cc:460] Webserver started at http://127.11.14.126:36513/ using document root <none> and password file <none>
I20260807 18:33:29.468807 16897 fs_manager.cc:359] Metadata directory not provided
I20260807 18:33:29.468873 16897 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:33:29.469089 16897 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:33:29.471248 16897 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786127579900523-11321-0/minicluster-data/master-0/data/instance:
uuid: "ee8670af25ce49b0a195856945c75419"
format_stamp: "Formatted at 2026-08-07 18:33:29 on dist-test-slave-6k22"
server_key: "c321a27f59f7e78ebf1a670d75fd3f8f"
server_key_iv: "b3142cd4766537ed3f3895e999991182"
server_key_version: "encryptionkey@0"
I20260807 18:33:29.471904 16897 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786127579900523-11321-0/minicluster-data/master-0/wal/instance:
uuid: "ee8670af25ce49b0a195856945c75419"
format_stamp: "Formatted at 2026-08-07 18:33:29 on dist-test-slave-6k22"
server_key: "c321a27f59f7e78ebf1a670d75fd3f8f"
server_key_iv: "b3142cd4766537ed3f3895e999991182"
server_key_version: "encryptionkey@0"
I20260807 18:33:29.475805 16897 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260807 18:33:29.478264 16915 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:33:29.479673 16897 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 18:33:29.479825 16897 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786127579900523-11321-0/minicluster-data/master-0/data,/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786127579900523-11321-0/minicluster-data/master-0/wal
uuid: "ee8670af25ce49b0a195856945c75419"
format_stamp: "Formatted at 2026-08-07 18:33:29 on dist-test-slave-6k22"
server_key: "c321a27f59f7e78ebf1a670d75fd3f8f"
server_key_iv: "b3142cd4766537ed3f3895e999991182"
server_key_version: "encryptionkey@0"
I20260807 18:33:29.479938 16897 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786127579900523-11321-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786127579900523-11321-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786127579900523-11321-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:33:29.496358 16897 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:33:29.497051 16897 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:33:29.497264 16897 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:33:29.504549 16897 rpc_server.cc:307] RPC server started. Bound to: 127.11.14.126:34369
I20260807 18:33:29.504554 16969 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.14.126:34369 every 8 connection(s)
I20260807 18:33:29.505501 16897 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786127579900523-11321-0/minicluster-data/master-0/data/info.pb
I20260807 18:33:29.508845 16970 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:33:29.513242 11321 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkz56XN/build/debug/bin/kudu as pid 16897
I20260807 18:33:29.513386 11321 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786127579900523-11321-0/minicluster-data/master-0/wal/instance
I20260807 18:33:29.513639 11321 external_mini_cluster.cc:1428] Setting key e90b885573ddcda495304d275fd715a5
I20260807 18:33:29.514917 16970 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ee8670af25ce49b0a195856945c75419: Bootstrap starting.
I20260807 18:33:29.517771 16970 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ee8670af25ce49b0a195856945c75419: Neither blocks nor log segments found. Creating new log.
I20260807 18:33:29.518730 16970 log.cc:824] T 00000000000000000000000000000000 P ee8670af25ce49b0a195856945c75419: Log is configured to *not* fsync() on all Append() calls
I20260807 18:33:29.521144 16970 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ee8670af25ce49b0a195856945c75419: No bootstrap required, opened a new log
I20260807 18:33:29.524690 16973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:29.514964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53146 (local address 127.11.14.126:34369)
0807 18:33:29.515759 (+ 795us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:29.515784 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:29.515829 (+ 45us) server_negotiation.cc:407] Connection header received
0807 18:33:29.516720 (+ 891us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:29.516744 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:29.517266 (+ 522us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:29.517623 (+ 357us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:33:29.518123 (+ 500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:29.519516 (+ 1393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:29.520609 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:29.521355 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:29.521573 (+ 218us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:29.522424 (+ 851us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:33:29.522430 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:33:29.522447 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:33:29.522958 (+ 511us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:33:29.523031 (+ 73us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:33:29.523060 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:33:29.523163 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:29.523375 (+ 212us) server_negotiation.cc:299] Negotiation successful
0807 18:33:29.523690 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":293,"thread_start_us":117,"threads_started":1}
I20260807 18:33:29.525806 16970 raft_consensus.cc:348] T 00000000000000000000000000000000 P ee8670af25ce49b0a195856945c75419 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee8670af25ce49b0a195856945c75419" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 34369 } }
I20260807 18:33:29.526015 16970 raft_consensus.cc:374] T 00000000000000000000000000000000 P ee8670af25ce49b0a195856945c75419 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:33:29.526085 16970 raft_consensus.cc:729] T 00000000000000000000000000000000 P ee8670af25ce49b0a195856945c75419 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee8670af25ce49b0a195856945c75419, State: Initialized, Role: FOLLOWER
I20260807 18:33:29.526640 16970 consensus_queue.cc:260] T 00000000000000000000000000000000 P ee8670af25ce49b0a195856945c75419 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee8670af25ce49b0a195856945c75419" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 34369 } }
I20260807 18:33:29.526809 16970 raft_consensus.cc:388] T 00000000000000000000000000000000 P ee8670af25ce49b0a195856945c75419 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:33:29.526865 16970 raft_consensus.cc:482] T 00000000000000000000000000000000 P ee8670af25ce49b0a195856945c75419 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:33:29.526952 16970 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ee8670af25ce49b0a195856945c75419 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:33:29.527645 11321 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 18:33:29.528106 16970 raft_consensus.cc:504] T 00000000000000000000000000000000 P ee8670af25ce49b0a195856945c75419 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee8670af25ce49b0a195856945c75419" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 34369 } }
I20260807 18:33:29.528517 16975 raft_consensus.cc:482] T 00000000000000000000000000000000 P ee8670af25ce49b0a195856945c75419 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:33:29.528517 16970 leader_election.cc:302] T 00000000000000000000000000000000 P ee8670af25ce49b0a195856945c75419 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ee8670af25ce49b0a195856945c75419; no voters:
I20260807 18:33:29.528687 16975 raft_consensus.cc:504] T 00000000000000000000000000000000 P ee8670af25ce49b0a195856945c75419 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee8670af25ce49b0a195856945c75419" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 34369 } }
I20260807 18:33:29.528983 16970 leader_election.cc:288] T 00000000000000000000000000000000 P ee8670af25ce49b0a195856945c75419 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:33:29.529153 16977 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ee8670af25ce49b0a195856945c75419 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:33:29.529258 16975 leader_election.cc:302] T 00000000000000000000000000000000 P ee8670af25ce49b0a195856945c75419 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ee8670af25ce49b0a195856945c75419; no voters:
I20260807 18:33:29.529381 16977 raft_consensus.cc:686] T 00000000000000000000000000000000 P ee8670af25ce49b0a195856945c75419 [term 1 LEADER]: Becoming Leader. State: Replica: ee8670af25ce49b0a195856945c75419, State: Running, Role: LEADER
I20260807 18:33:29.529438 16975 leader_election.cc:288] T 00000000000000000000000000000000 P ee8670af25ce49b0a195856945c75419 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers
I20260807 18:33:29.529672 16977 consensus_queue.cc:237] T 00000000000000000000000000000000 P ee8670af25ce49b0a195856945c75419 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee8670af25ce49b0a195856945c75419" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 34369 } }
I20260807 18:33:29.530275 16970 sys_catalog.cc:564] T 00000000000000000000000000000000 P ee8670af25ce49b0a195856945c75419 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:33:29.531109 16975 sys_catalog.cc:455] T 00000000000000000000000000000000 P ee8670af25ce49b0a195856945c75419 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ee8670af25ce49b0a195856945c75419" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee8670af25ce49b0a195856945c75419" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 34369 } } }
I20260807 18:33:29.531184 16979 sys_catalog.cc:455] T 00000000000000000000000000000000 P ee8670af25ce49b0a195856945c75419 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ee8670af25ce49b0a195856945c75419. Latest consensus state: current_term: 1 leader_uuid: "ee8670af25ce49b0a195856945c75419" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee8670af25ce49b0a195856945c75419" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 34369 } } }
I20260807 18:33:29.531250 16975 sys_catalog.cc:458] T 00000000000000000000000000000000 P ee8670af25ce49b0a195856945c75419 [sys.catalog]: This master's current role is: LEADER
I20260807 18:33:29.531286 16979 sys_catalog.cc:458] T 00000000000000000000000000000000 P ee8670af25ce49b0a195856945c75419 [sys.catalog]: This master's current role is: LEADER
I20260807 18:33:29.531945 16987 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:33:29.534721 16987 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:33:29.540201 16987 catalog_manager.cc:1347] Generated new cluster ID: 74d6cb29b5404269a035a53e1d280510
I20260807 18:33:29.540274 16987 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:33:29.554832 16987 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:33:29.556139 16987 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:33:29.571429 16987 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ee8670af25ce49b0a195856945c75419: Generated new TSK 0
I20260807 18:33:29.572283 16987 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260807 18:33:29.609887 17004 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260807 18:33:29.610080 17004 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260807 18:33:29.610720 16973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:29.590394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53156 (local address 127.11.14.126:34369)
0807 18:33:29.590563 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:29.590569 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:29.590835 (+ 266us) server_negotiation.cc:407] Connection header received
0807 18:33:29.591069 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:29.591073 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:29.591126 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:29.591249 (+ 123us) server_negotiation.cc:226] Negotiated authn=JWT
0807 18:33:29.591605 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:29.592247 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:29.593517 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:29.593765 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:29.593851 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:29.594270 (+ 419us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0807 18:33:29.610331 (+ 16061us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33757/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0807 18:33:29.610592 (+ 261us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:53156: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33757/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":67}
W20260807 18:33:29.610834 16973 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:53156: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33757/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260807 18:33:29.612460 17003 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:33:29.590317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:34369 (local address 127.0.0.1:53156)
0807 18:33:29.590731 (+ 414us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:29.590747 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:29.590893 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:29.591286 (+ 393us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:29.591289 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:29.591298 (+ 9us) client_negotiation.cc:189] Negotiated authn=JWT
0807 18:33:29.591438 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:29.591444 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:29.592378 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:29.592382 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:29.593391 (+ 1009us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:29.593402 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:29.593892 (+ 490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:29.593897 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:29.593996 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:29.594055 (+ 59us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0807 18:33:29.610639 (+ 16584us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33757/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0807 18:33:29.610934 (+ 295us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.11.14.126:34369: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33757/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":147,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.11.14.126:34369: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33757/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260807 18:33:29.613709 11321 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkz56XN/build/debug/bin/kudu with pid 16897
2026-08-07T18:33:29Z chronyd exiting
I20260807 18:33:29.654091 11321 test_util.cc:175] -----------------------------------------------
I20260807 18:33:29.654204 11321 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786127579900523-11321-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (342 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260807 18:33:29.655413 11321 mini_oidc.cc:152] Starting JWKS server
I20260807 18:33:29.656360 11321 webserver.cc:460] Webserver started at https://127.0.0.1:37821/ using document root <none> and password file <none>
I20260807 18:33:29.656441 11321 mini_oidc.cc:189] Starting OIDC Discovery server
I20260807 18:33:29.656713 11321 webserver.cc:460] Webserver started at http://127.0.0.1:34717/ using document root <none> and password file <none>
2026-08-07T18:33:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:33:29Z Disabled control of system clock
I20260807 18:33:29.667111 11321 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkz56XN/build/debug/bin/kudu
/tmp/dist-test-taskkz56XN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786127579900523-11321-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786127579900523-11321-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786127579900523-11321-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786127579900523-11321-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.14.126:40757
--webserver_interface=127.11.14.126
--webserver_port=0
--builtin_ntp_servers=127.11.14.84:34127
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.14.126:40757
--enable_jwt_token_auth=true
--jwks_url=https://localhost:37821/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786127579900523-11321-0/testchainca.cert with env {}
W20260807 18:33:29.785085 17010 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:33:29.785355 17010 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:33:29.785395 17010 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:33:29.789386 17010 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:33:29.789444 17010 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:33:29.789460 17010 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:33:29.789477 17010 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:33:29.789492 17010 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260807 18:33:29.789507 17010 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260807 18:33:29.789521 17010 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:37821/jwks.json
I20260807 18:33:29.795217 17010 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.14.84:34127
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786127579900523-11321-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786127579900523-11321-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.14.126:40757
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.14.126:40757
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:37821/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786127579900523-11321-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.14.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786127579900523-11321-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17010
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786127579900523-11321-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:33:29.796564 17010 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:33:29.797583 17010 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:33:29.806650 17018 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:33:29.806648 17017 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:33:29.806823 17010 server_base.cc:1034] running on GCE node
W20260807 18:33:29.806648 17020 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:33:29.807729 17010 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:33:29.808902 17010 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:33:29.810097 17010 hybrid_clock.cc:648] HybridClock initialized: now 1786127609810082 us; error 29 us; skew 500 ppm
I20260807 18:33:29.812131 17010 webserver.cc:460] Webserver started at http://127.11.14.126:37557/ using document root <none> and password file <none>
I20260807 18:33:29.812747 17010 fs_manager.cc:359] Metadata directory not provided
I20260807 18:33:29.812845 17010 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:33:29.813076 17010 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:33:29.815129 17010 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786127579900523-11321-0/minicluster-data/master-0/data/instance:
uuid: "2d1881ec041545b0b41f28a82b960c27"
format_stamp: "Formatted at 2026-08-07 18:33:29 on dist-test-slave-6k22"
server_key: "e180ee397d22cc23412be229e41a1a85"
server_key_iv: "c39c8c5f470a32b13d58f767fa21640e"
server_key_version: "encryptionkey@0"
I20260807 18:33:29.815793 17010 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786127579900523-11321-0/minicluster-data/master-0/wal/instance:
uuid: "2d1881ec041545b0b41f28a82b960c27"
format_stamp: "Formatted at 2026-08-07 18:33:29 on dist-test-slave-6k22"
server_key: "e180ee397d22cc23412be229e41a1a85"
server_key_iv: "c39c8c5f470a32b13d58f767fa21640e"
server_key_version: "encryptionkey@0"
I20260807 18:33:29.819500 17010 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260807 18:33:29.822160 17026 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:33:29.823493 17010 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260807 18:33:29.823719 17010 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786127579900523-11321-0/minicluster-data/master-0/data,/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786127579900523-11321-0/minicluster-data/master-0/wal
uuid: "2d1881ec041545b0b41f28a82b960c27"
format_stamp: "Formatted at 2026-08-07 18:33:29 on dist-test-slave-6k22"
server_key: "e180ee397d22cc23412be229e41a1a85"
server_key_iv: "c39c8c5f470a32b13d58f767fa21640e"
server_key_version: "encryptionkey@0"
I20260807 18:33:29.823853 17010 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786127579900523-11321-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786127579900523-11321-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786127579900523-11321-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:33:29.847054 17010 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:33:29.847903 17010 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:33:29.848111 17010 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:33:29.856024 17010 rpc_server.cc:307] RPC server started. Bound to: 127.11.14.126:40757
I20260807 18:33:29.856041 17081 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.14.126:40757 every 8 connection(s)
I20260807 18:33:29.857286 17010 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786127579900523-11321-0/minicluster-data/master-0/data/info.pb
I20260807 18:33:29.860770 17082 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:33:29.863881 11321 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkz56XN/build/debug/bin/kudu as pid 17010
I20260807 18:33:29.864037 11321 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786127579900523-11321-0/minicluster-data/master-0/wal/instance
I20260807 18:33:29.864357 11321 external_mini_cluster.cc:1428] Setting key cbaac4135708e6096b01c803ce3030af
I20260807 18:33:29.867218 17082 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d1881ec041545b0b41f28a82b960c27: Bootstrap starting.
I20260807 18:33:29.869536 17082 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2d1881ec041545b0b41f28a82b960c27: Neither blocks nor log segments found. Creating new log.
I20260807 18:33:29.870458 17082 log.cc:824] T 00000000000000000000000000000000 P 2d1881ec041545b0b41f28a82b960c27: Log is configured to *not* fsync() on all Append() calls
I20260807 18:33:29.872987 17082 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d1881ec041545b0b41f28a82b960c27: No bootstrap required, opened a new log
I20260807 18:33:29.874622 17085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:29.865823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42108 (local address 127.11.14.126:40757)
0807 18:33:29.866520 (+ 697us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:29.866540 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:29.866576 (+ 36us) server_negotiation.cc:407] Connection header received
0807 18:33:29.867236 (+ 660us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:29.867254 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:29.867784 (+ 530us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:29.868121 (+ 337us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:33:29.868578 (+ 457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:29.869540 (+ 962us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:29.870824 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:29.871668 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:29.871808 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:29.872691 (+ 883us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:33:29.872696 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:33:29.872710 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:33:29.873190 (+ 480us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:33:29.873250 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:33:29.873273 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:33:29.873359 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:29.873539 (+ 180us) server_negotiation.cc:299] Negotiation successful
0807 18:33:29.873864 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":325,"thread_start_us":141,"threads_started":1}
I20260807 18:33:29.876885 17082 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2d1881ec041545b0b41f28a82b960c27 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d1881ec041545b0b41f28a82b960c27" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 40757 } }
I20260807 18:33:29.877162 17082 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2d1881ec041545b0b41f28a82b960c27 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:33:29.877280 17082 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2d1881ec041545b0b41f28a82b960c27 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d1881ec041545b0b41f28a82b960c27, State: Initialized, Role: FOLLOWER
I20260807 18:33:29.877751 17082 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2d1881ec041545b0b41f28a82b960c27 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d1881ec041545b0b41f28a82b960c27" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 40757 } }
I20260807 18:33:29.877851 11321 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 18:33:29.877941 17082 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2d1881ec041545b0b41f28a82b960c27 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:33:29.878002 17082 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2d1881ec041545b0b41f28a82b960c27 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:33:29.878094 17082 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2d1881ec041545b0b41f28a82b960c27 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:33:29.879274 17082 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2d1881ec041545b0b41f28a82b960c27 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d1881ec041545b0b41f28a82b960c27" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 40757 } }
I20260807 18:33:29.879689 17082 leader_election.cc:302] T 00000000000000000000000000000000 P 2d1881ec041545b0b41f28a82b960c27 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2d1881ec041545b0b41f28a82b960c27; no voters:
I20260807 18:33:29.880033 17082 leader_election.cc:288] T 00000000000000000000000000000000 P 2d1881ec041545b0b41f28a82b960c27 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:33:29.880138 17087 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2d1881ec041545b0b41f28a82b960c27 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:33:29.880419 17087 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2d1881ec041545b0b41f28a82b960c27 [term 1 LEADER]: Becoming Leader. State: Replica: 2d1881ec041545b0b41f28a82b960c27, State: Running, Role: LEADER
I20260807 18:33:29.880769 17087 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2d1881ec041545b0b41f28a82b960c27 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d1881ec041545b0b41f28a82b960c27" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 40757 } }
I20260807 18:33:29.881309 17082 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2d1881ec041545b0b41f28a82b960c27 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:33:29.882484 17088 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d1881ec041545b0b41f28a82b960c27 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2d1881ec041545b0b41f28a82b960c27" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d1881ec041545b0b41f28a82b960c27" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 40757 } } }
I20260807 18:33:29.882665 17088 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d1881ec041545b0b41f28a82b960c27 [sys.catalog]: This master's current role is: LEADER
I20260807 18:33:29.882953 17089 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d1881ec041545b0b41f28a82b960c27 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2d1881ec041545b0b41f28a82b960c27. Latest consensus state: current_term: 1 leader_uuid: "2d1881ec041545b0b41f28a82b960c27" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d1881ec041545b0b41f28a82b960c27" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 40757 } } }
I20260807 18:33:29.883062 17089 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d1881ec041545b0b41f28a82b960c27 [sys.catalog]: This master's current role is: LEADER
I20260807 18:33:29.883072 17098 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:33:29.886292 17098 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:33:29.892479 17098 catalog_manager.cc:1347] Generated new cluster ID: a29b74375ce24bfb9ca3ca502611ecf1
I20260807 18:33:29.892583 17098 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:33:29.905110 17098 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:33:29.906112 17098 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:33:29.910874 17098 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2d1881ec041545b0b41f28a82b960c27: Generated new TSK 0
I20260807 18:33:29.911628 17098 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260807 18:33:29.945935 17115 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260807 18:33:29.946043 17115 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260807 18:33:29.946718 17085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:29.923133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42110 (local address 127.11.14.126:40757)
0807 18:33:29.923266 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:29.923271 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:29.923451 (+ 180us) server_negotiation.cc:407] Connection header received
0807 18:33:29.923678 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:29.923681 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:29.923729 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:29.923814 (+ 85us) server_negotiation.cc:226] Negotiated authn=JWT
0807 18:33:29.924097 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:29.924624 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:29.925883 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:29.926107 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:29.926181 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:29.926575 (+ 394us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0807 18:33:29.946321 (+ 19746us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37821/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0807 18:33:29.946584 (+ 263us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:42110: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37821/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":54}
W20260807 18:33:29.946841 17085 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:42110: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37821/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260807 18:33:29.946933 17114 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:33:29.923004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:40757 (local address 127.0.0.1:42110)
0807 18:33:29.923345 (+ 341us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:29.923361 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:29.923489 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:29.923843 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:29.923847 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:29.923856 (+ 9us) client_negotiation.cc:189] Negotiated authn=JWT
0807 18:33:29.923987 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:29.923994 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:29.924736 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:29.924740 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:29.925765 (+ 1025us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:29.925776 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:29.926221 (+ 445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:29.926225 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:29.926336 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:29.926367 (+ 31us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0807 18:33:29.946549 (+ 20182us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37821/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0807 18:33:29.946742 (+ 193us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.11.14.126:40757: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37821/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":100,"threads_started":1}
I20260807 18:33:29.947382 11321 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkz56XN/build/debug/bin/kudu with pid 17010
2026-08-07T18:33:29Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (334 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260807 18:33:29.990006 11321 mini_oidc.cc:152] Starting JWKS server
I20260807 18:33:29.990890 11321 webserver.cc:460] Webserver started at https://127.0.0.1:39827/ using document root <none> and password file <none>
I20260807 18:33:29.990983 11321 mini_oidc.cc:189] Starting OIDC Discovery server
I20260807 18:33:29.991237 11321 webserver.cc:460] Webserver started at http://127.0.0.1:43327/ using document root <none> and password file <none>
2026-08-07T18:33:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:33:29Z Disabled control of system clock
I20260807 18:33:30.001596 11321 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkz56XN/build/debug/bin/kudu
/tmp/dist-test-taskkz56XN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786127579900523-11321-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786127579900523-11321-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786127579900523-11321-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786127579900523-11321-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.14.126:43789
--webserver_interface=127.11.14.126
--webserver_port=0
--builtin_ntp_servers=127.11.14.84:41359
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.14.126:43789
--enable_jwt_token_auth=true
--jwks_url=https://localhost:39827/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260807 18:33:30.116554 17123 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:33:30.116850 17123 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:33:30.116891 17123 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:33:30.121017 17123 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:33:30.121085 17123 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:33:30.121101 17123 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:33:30.121119 17123 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:33:30.121133 17123 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260807 18:33:30.121150 17123 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260807 18:33:30.121198 17123 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:39827/jwks.json
I20260807 18:33:30.125943 17123 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.14.84:41359
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786127579900523-11321-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786127579900523-11321-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.14.126:43789
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.14.126:43789
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:39827/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786127579900523-11321-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.14.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17123
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786127579900523-11321-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:33:30.127122 17123 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:33:30.128674 17123 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:33:30.137523 17128 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:33:30.137513 17129 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:33:30.137526 17132 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:33:30.137957 17123 server_base.cc:1034] running on GCE node
I20260807 18:33:30.138556 17123 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:33:30.139798 17123 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:33:30.141026 17123 hybrid_clock.cc:648] HybridClock initialized: now 1786127610140996 us; error 34 us; skew 500 ppm
I20260807 18:33:30.142968 17123 webserver.cc:460] Webserver started at http://127.11.14.126:36499/ using document root <none> and password file <none>
I20260807 18:33:30.143640 17123 fs_manager.cc:359] Metadata directory not provided
I20260807 18:33:30.143743 17123 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:33:30.144007 17123 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:33:30.146106 17123 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786127579900523-11321-0/minicluster-data/master-0/data/instance:
uuid: "f04b5af5554a49a9bb5a6a767b58852e"
format_stamp: "Formatted at 2026-08-07 18:33:30 on dist-test-slave-6k22"
server_key: "608eeb2f17a33f25f3d74683a5720e20"
server_key_iv: "da9a7d8e12fd02b8f59c1e59d47c9d56"
server_key_version: "encryptionkey@0"
I20260807 18:33:30.146698 17123 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786127579900523-11321-0/minicluster-data/master-0/wal/instance:
uuid: "f04b5af5554a49a9bb5a6a767b58852e"
format_stamp: "Formatted at 2026-08-07 18:33:30 on dist-test-slave-6k22"
server_key: "608eeb2f17a33f25f3d74683a5720e20"
server_key_iv: "da9a7d8e12fd02b8f59c1e59d47c9d56"
server_key_version: "encryptionkey@0"
I20260807 18:33:30.150512 17123 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260807 18:33:30.153162 17138 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:33:30.154402 17123 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 18:33:30.154548 17123 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786127579900523-11321-0/minicluster-data/master-0/data,/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786127579900523-11321-0/minicluster-data/master-0/wal
uuid: "f04b5af5554a49a9bb5a6a767b58852e"
format_stamp: "Formatted at 2026-08-07 18:33:30 on dist-test-slave-6k22"
server_key: "608eeb2f17a33f25f3d74683a5720e20"
server_key_iv: "da9a7d8e12fd02b8f59c1e59d47c9d56"
server_key_version: "encryptionkey@0"
I20260807 18:33:30.154663 17123 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786127579900523-11321-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786127579900523-11321-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786127579900523-11321-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:33:30.183753 17123 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:33:30.184525 17123 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:33:30.184738 17123 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:33:30.193069 17123 rpc_server.cc:307] RPC server started. Bound to: 127.11.14.126:43789
I20260807 18:33:30.193091 17195 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.14.126:43789 every 8 connection(s)
I20260807 18:33:30.194032 17123 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786127579900523-11321-0/minicluster-data/master-0/data/info.pb
I20260807 18:33:30.197140 17196 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:33:30.200115 11321 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkz56XN/build/debug/bin/kudu as pid 17123
I20260807 18:33:30.200253 11321 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786127579900523-11321-0/minicluster-data/master-0/wal/instance
I20260807 18:33:30.200556 11321 external_mini_cluster.cc:1428] Setting key 4aa4c1053d89150fd9fd6ca98f58240a
I20260807 18:33:30.204205 17196 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f04b5af5554a49a9bb5a6a767b58852e: Bootstrap starting.
I20260807 18:33:30.206475 17196 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f04b5af5554a49a9bb5a6a767b58852e: Neither blocks nor log segments found. Creating new log.
I20260807 18:33:30.207253 17196 log.cc:824] T 00000000000000000000000000000000 P f04b5af5554a49a9bb5a6a767b58852e: Log is configured to *not* fsync() on all Append() calls
I20260807 18:33:30.209801 17196 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f04b5af5554a49a9bb5a6a767b58852e: No bootstrap required, opened a new log
I20260807 18:33:30.210358 17199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:30.201876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60164 (local address 127.11.14.126:43789)
0807 18:33:30.202555 (+ 679us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:30.202571 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:30.202609 (+ 38us) server_negotiation.cc:407] Connection header received
0807 18:33:30.203283 (+ 674us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:30.203301 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:30.203876 (+ 575us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:30.204226 (+ 350us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:33:30.204679 (+ 453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:30.205658 (+ 979us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:30.206585 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:30.207078 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:30.207267 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:30.208213 (+ 946us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:33:30.208223 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:33:30.208243 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:33:30.208728 (+ 485us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:33:30.208802 (+ 74us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:33:30.208829 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:33:30.208941 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:30.209139 (+ 198us) server_negotiation.cc:299] Negotiation successful
0807 18:33:30.209405 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":284,"thread_start_us":115,"threads_started":1}
I20260807 18:33:30.213351 11321 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 18:33:30.214112 17196 raft_consensus.cc:348] T 00000000000000000000000000000000 P f04b5af5554a49a9bb5a6a767b58852e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f04b5af5554a49a9bb5a6a767b58852e" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 43789 } }
I20260807 18:33:30.214363 17196 raft_consensus.cc:374] T 00000000000000000000000000000000 P f04b5af5554a49a9bb5a6a767b58852e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:33:30.214428 17196 raft_consensus.cc:729] T 00000000000000000000000000000000 P f04b5af5554a49a9bb5a6a767b58852e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f04b5af5554a49a9bb5a6a767b58852e, State: Initialized, Role: FOLLOWER
I20260807 18:33:30.214865 17196 consensus_queue.cc:260] T 00000000000000000000000000000000 P f04b5af5554a49a9bb5a6a767b58852e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f04b5af5554a49a9bb5a6a767b58852e" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 43789 } }
I20260807 18:33:30.215042 17196 raft_consensus.cc:388] T 00000000000000000000000000000000 P f04b5af5554a49a9bb5a6a767b58852e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:33:30.215126 17196 raft_consensus.cc:482] T 00000000000000000000000000000000 P f04b5af5554a49a9bb5a6a767b58852e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:33:30.215243 17196 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f04b5af5554a49a9bb5a6a767b58852e [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:33:30.216228 17196 raft_consensus.cc:504] T 00000000000000000000000000000000 P f04b5af5554a49a9bb5a6a767b58852e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f04b5af5554a49a9bb5a6a767b58852e" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 43789 } }
I20260807 18:33:30.216583 17196 leader_election.cc:302] T 00000000000000000000000000000000 P f04b5af5554a49a9bb5a6a767b58852e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f04b5af5554a49a9bb5a6a767b58852e; no voters:
I20260807 18:33:30.216897 17196 leader_election.cc:288] T 00000000000000000000000000000000 P f04b5af5554a49a9bb5a6a767b58852e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:33:30.217028 17201 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f04b5af5554a49a9bb5a6a767b58852e [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:33:30.217269 17201 raft_consensus.cc:686] T 00000000000000000000000000000000 P f04b5af5554a49a9bb5a6a767b58852e [term 1 LEADER]: Becoming Leader. State: Replica: f04b5af5554a49a9bb5a6a767b58852e, State: Running, Role: LEADER
I20260807 18:33:30.217691 17201 consensus_queue.cc:237] T 00000000000000000000000000000000 P f04b5af5554a49a9bb5a6a767b58852e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f04b5af5554a49a9bb5a6a767b58852e" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 43789 } }
I20260807 18:33:30.218160 17196 sys_catalog.cc:564] T 00000000000000000000000000000000 P f04b5af5554a49a9bb5a6a767b58852e [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:33:30.219442 17203 sys_catalog.cc:455] T 00000000000000000000000000000000 P f04b5af5554a49a9bb5a6a767b58852e [sys.catalog]: SysCatalogTable state changed. Reason: New leader f04b5af5554a49a9bb5a6a767b58852e. Latest consensus state: current_term: 1 leader_uuid: "f04b5af5554a49a9bb5a6a767b58852e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f04b5af5554a49a9bb5a6a767b58852e" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 43789 } } }
I20260807 18:33:30.219594 17203 sys_catalog.cc:458] T 00000000000000000000000000000000 P f04b5af5554a49a9bb5a6a767b58852e [sys.catalog]: This master's current role is: LEADER
I20260807 18:33:30.219941 17202 sys_catalog.cc:455] T 00000000000000000000000000000000 P f04b5af5554a49a9bb5a6a767b58852e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f04b5af5554a49a9bb5a6a767b58852e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f04b5af5554a49a9bb5a6a767b58852e" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 43789 } } }
I20260807 18:33:30.220027 17212 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:33:30.220113 17202 sys_catalog.cc:458] T 00000000000000000000000000000000 P f04b5af5554a49a9bb5a6a767b58852e [sys.catalog]: This master's current role is: LEADER
I20260807 18:33:30.223626 17212 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:33:30.229059 17212 catalog_manager.cc:1347] Generated new cluster ID: 047bb9c156d64dc88e3ddeecd8cbd7a1
I20260807 18:33:30.229221 17212 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:33:30.248472 17212 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:33:30.249563 17212 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:33:30.261031 17212 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f04b5af5554a49a9bb5a6a767b58852e: Generated new TSK 0
I20260807 18:33:30.261829 17212 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:33:30.280616 17199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:30.274237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60172 (local address 127.11.14.126:43789)
0807 18:33:30.274429 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:30.274435 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:30.274799 (+ 364us) server_negotiation.cc:407] Connection header received
0807 18:33:30.274976 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:30.274980 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:30.275045 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:30.275182 (+ 137us) server_negotiation.cc:226] Negotiated authn=JWT
0807 18:33:30.275420 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:30.276307 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:30.277346 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:30.277698 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:30.277803 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:30.278138 (+ 335us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0807 18:33:30.279821 (+ 1683us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39827/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0807 18:33:30.280077 (+ 256us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:60172: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39827/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":76}
W20260807 18:33:30.280707 17199 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:60172: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39827/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260807 18:33:30.282280 17227 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:33:30.274178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:43789 (local address 127.0.0.1:60172)
0807 18:33:30.274616 (+ 438us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:30.274705 (+ 89us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:30.274824 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:30.275189 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:30.275192 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:30.275199 (+ 7us) client_negotiation.cc:189] Negotiated authn=JWT
0807 18:33:30.275309 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:30.275314 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:30.276437 (+ 1123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:30.276440 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:30.277222 (+ 782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:30.277232 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:30.277831 (+ 599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:30.277835 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:30.277907 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:30.277932 (+ 25us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0807 18:33:30.280011 (+ 2079us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39827/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0807 18:33:30.282084 (+ 2073us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.11.14.126:43789: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39827/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":362,"mutex_wait_us":1848,"thread_start_us":151,"threads_started":1}
I20260807 18:33:30.283073 11321 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkz56XN/build/debug/bin/kudu with pid 17123
E20260807 18:33:30.297766 17228 webserver.cc:533] Webserver: sslize failed:
E20260807 18:33:30.297863 17228 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-07T18:33:30Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (339 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-07T18:33:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:33:30Z Disabled control of system clock
I20260807 18:33:30.340569 11321 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkz56XN/build/debug/bin/kudu
/tmp/dist-test-taskkz56XN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786127579900523-11321-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786127579900523-11321-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786127579900523-11321-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786127579900523-11321-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.14.126:37181
--webserver_interface=127.11.14.126
--webserver_port=0
--builtin_ntp_servers=127.11.14.84:38467
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.14.126:37181
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786127579900523-11321-0/testchainca.cert with env {}
W20260807 18:33:30.459004 17232 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:33:30.459291 17232 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:33:30.459332 17232 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:33:30.463400 17232 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:33:30.463476 17232 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:33:30.463492 17232 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:33:30.463508 17232 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:33:30.463524 17232 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260807 18:33:30.463541 17232 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260807 18:33:30.463601 17232 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260807 18:33:30.469364 17232 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.14.84:38467
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786127579900523-11321-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786127579900523-11321-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.14.126:37181
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.14.126:37181
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786127579900523-11321-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.14.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786127579900523-11321-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17232
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786127579900523-11321-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:33:30.470490 17232 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:33:30.471642 17232 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:33:30.480517 17241 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:33:30.480552 17239 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:33:30.480552 17237 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:33:30.480777 17232 server_base.cc:1034] running on GCE node
I20260807 18:33:30.481326 17232 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:33:30.482462 17232 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:33:30.483634 17232 hybrid_clock.cc:648] HybridClock initialized: now 1786127610483615 us; error 32 us; skew 500 ppm
I20260807 18:33:30.485639 17232 webserver.cc:460] Webserver started at http://127.11.14.126:39763/ using document root <none> and password file <none>
I20260807 18:33:30.486253 17232 fs_manager.cc:359] Metadata directory not provided
I20260807 18:33:30.486348 17232 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:33:30.486588 17232 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:33:30.488804 17232 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786127579900523-11321-0/minicluster-data/master-0/data/instance:
uuid: "ee1c185e1a274da499f9d94b23bfff89"
format_stamp: "Formatted at 2026-08-07 18:33:30 on dist-test-slave-6k22"
server_key: "99d9ffc84bd485c4f03c66d3451da246"
server_key_iv: "2777ed83dffad6030ac3fce558139f25"
server_key_version: "encryptionkey@0"
I20260807 18:33:30.489401 17232 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786127579900523-11321-0/minicluster-data/master-0/wal/instance:
uuid: "ee1c185e1a274da499f9d94b23bfff89"
format_stamp: "Formatted at 2026-08-07 18:33:30 on dist-test-slave-6k22"
server_key: "99d9ffc84bd485c4f03c66d3451da246"
server_key_iv: "2777ed83dffad6030ac3fce558139f25"
server_key_version: "encryptionkey@0"
I20260807 18:33:30.493389 17232 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 18:33:30.495960 17247 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:33:30.497340 17232 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:33:30.497517 17232 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786127579900523-11321-0/minicluster-data/master-0/data,/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786127579900523-11321-0/minicluster-data/master-0/wal
uuid: "ee1c185e1a274da499f9d94b23bfff89"
format_stamp: "Formatted at 2026-08-07 18:33:30 on dist-test-slave-6k22"
server_key: "99d9ffc84bd485c4f03c66d3451da246"
server_key_iv: "2777ed83dffad6030ac3fce558139f25"
server_key_version: "encryptionkey@0"
I20260807 18:33:30.497721 17232 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786127579900523-11321-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786127579900523-11321-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786127579900523-11321-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:33:30.509616 17232 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:33:30.510357 17232 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:33:30.510568 17232 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:33:30.518487 17232 rpc_server.cc:307] RPC server started. Bound to: 127.11.14.126:37181
I20260807 18:33:30.518510 17301 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.14.126:37181 every 8 connection(s)
I20260807 18:33:30.519541 17232 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786127579900523-11321-0/minicluster-data/master-0/data/info.pb
I20260807 18:33:30.522498 17302 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:33:30.527354 11321 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkz56XN/build/debug/bin/kudu as pid 17232
I20260807 18:33:30.527485 11321 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786127579900523-11321-0/minicluster-data/master-0/wal/instance
I20260807 18:33:30.527842 11321 external_mini_cluster.cc:1428] Setting key b3f3d5e261feafeeda164cf96f37886c
I20260807 18:33:30.528488 17302 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ee1c185e1a274da499f9d94b23bfff89: Bootstrap starting.
I20260807 18:33:30.531032 17302 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ee1c185e1a274da499f9d94b23bfff89: Neither blocks nor log segments found. Creating new log.
I20260807 18:33:30.531852 17302 log.cc:824] T 00000000000000000000000000000000 P ee1c185e1a274da499f9d94b23bfff89: Log is configured to *not* fsync() on all Append() calls
I20260807 18:33:30.533808 17302 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ee1c185e1a274da499f9d94b23bfff89: No bootstrap required, opened a new log
I20260807 18:33:30.537396 17305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:30.529227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45798 (local address 127.11.14.126:37181)
0807 18:33:30.529920 (+ 693us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:30.529935 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:30.529972 (+ 37us) server_negotiation.cc:407] Connection header received
0807 18:33:30.530614 (+ 642us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:30.530629 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:30.531029 (+ 400us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:30.531419 (+ 390us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:33:30.531901 (+ 482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:30.533122 (+ 1221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:30.534081 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:30.534559 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:30.534737 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:30.535654 (+ 917us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:33:30.535659 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:33:30.535672 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:33:30.536064 (+ 392us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:33:30.536127 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:33:30.536148 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:33:30.536274 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:30.536407 (+ 133us) server_negotiation.cc:299] Negotiation successful
0807 18:33:30.536663 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":305,"thread_start_us":139,"threads_started":1}
I20260807 18:33:30.537745 17302 raft_consensus.cc:348] T 00000000000000000000000000000000 P ee1c185e1a274da499f9d94b23bfff89 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee1c185e1a274da499f9d94b23bfff89" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 37181 } }
I20260807 18:33:30.538021 17302 raft_consensus.cc:374] T 00000000000000000000000000000000 P ee1c185e1a274da499f9d94b23bfff89 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:33:30.538081 17302 raft_consensus.cc:729] T 00000000000000000000000000000000 P ee1c185e1a274da499f9d94b23bfff89 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee1c185e1a274da499f9d94b23bfff89, State: Initialized, Role: FOLLOWER
I20260807 18:33:30.538565 17302 consensus_queue.cc:260] T 00000000000000000000000000000000 P ee1c185e1a274da499f9d94b23bfff89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee1c185e1a274da499f9d94b23bfff89" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 37181 } }
I20260807 18:33:30.538698 17302 raft_consensus.cc:388] T 00000000000000000000000000000000 P ee1c185e1a274da499f9d94b23bfff89 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:33:30.538790 17302 raft_consensus.cc:482] T 00000000000000000000000000000000 P ee1c185e1a274da499f9d94b23bfff89 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:33:30.538910 17302 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ee1c185e1a274da499f9d94b23bfff89 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:33:30.540076 17302 raft_consensus.cc:504] T 00000000000000000000000000000000 P ee1c185e1a274da499f9d94b23bfff89 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee1c185e1a274da499f9d94b23bfff89" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 37181 } }
I20260807 18:33:30.540479 11321 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 18:33:30.540472 17302 leader_election.cc:302] T 00000000000000000000000000000000 P ee1c185e1a274da499f9d94b23bfff89 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ee1c185e1a274da499f9d94b23bfff89; no voters:
I20260807 18:33:30.540771 17302 leader_election.cc:288] T 00000000000000000000000000000000 P ee1c185e1a274da499f9d94b23bfff89 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:33:30.540912 17308 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ee1c185e1a274da499f9d94b23bfff89 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:33:30.541139 17308 raft_consensus.cc:686] T 00000000000000000000000000000000 P ee1c185e1a274da499f9d94b23bfff89 [term 1 LEADER]: Becoming Leader. State: Replica: ee1c185e1a274da499f9d94b23bfff89, State: Running, Role: LEADER
I20260807 18:33:30.541461 17308 consensus_queue.cc:237] T 00000000000000000000000000000000 P ee1c185e1a274da499f9d94b23bfff89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee1c185e1a274da499f9d94b23bfff89" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 37181 } }
I20260807 18:33:30.542096 17302 sys_catalog.cc:564] T 00000000000000000000000000000000 P ee1c185e1a274da499f9d94b23bfff89 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:33:30.543270 17310 sys_catalog.cc:455] T 00000000000000000000000000000000 P ee1c185e1a274da499f9d94b23bfff89 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ee1c185e1a274da499f9d94b23bfff89. Latest consensus state: current_term: 1 leader_uuid: "ee1c185e1a274da499f9d94b23bfff89" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee1c185e1a274da499f9d94b23bfff89" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 37181 } } }
I20260807 18:33:30.543396 17310 sys_catalog.cc:458] T 00000000000000000000000000000000 P ee1c185e1a274da499f9d94b23bfff89 [sys.catalog]: This master's current role is: LEADER
I20260807 18:33:30.543810 17318 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:33:30.543807 17309 sys_catalog.cc:455] T 00000000000000000000000000000000 P ee1c185e1a274da499f9d94b23bfff89 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ee1c185e1a274da499f9d94b23bfff89" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee1c185e1a274da499f9d94b23bfff89" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 37181 } } }
I20260807 18:33:30.544479 17309 sys_catalog.cc:458] T 00000000000000000000000000000000 P ee1c185e1a274da499f9d94b23bfff89 [sys.catalog]: This master's current role is: LEADER
I20260807 18:33:30.546749 17318 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:33:30.552768 17318 catalog_manager.cc:1347] Generated new cluster ID: f1ea486729fb4cc5892a77debb4425cc
I20260807 18:33:30.552841 17318 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:33:30.581697 17318 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:33:30.582954 17318 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:33:30.591884 17318 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ee1c185e1a274da499f9d94b23bfff89: Generated new TSK 0
I20260807 18:33:30.592727 17318 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:33:31.070657 17305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:30.601155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45814 (local address 127.11.14.126:37181)
0807 18:33:30.601301 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:30.601306 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:30.601566 (+ 260us) server_negotiation.cc:407] Connection header received
0807 18:33:30.601724 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:30.601727 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:30.601786 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:30.601997 (+ 211us) server_negotiation.cc:226] Negotiated authn=JWT
0807 18:33:30.602276 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:30.603042 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:30.604395 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:30.604628 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:30.604724 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:30.605057 (+ 333us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0807 18:33:31.070236 (+465179us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0807 18:33:31.070527 (+ 291us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:45814: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":54}
W20260807 18:33:31.070784 17305 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:45814: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260807 18:33:31.070960 17334 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:33:30.601097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:37181 (local address 127.0.0.1:45814)
0807 18:33:30.601433 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:30.601450 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:30.601575 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:30.601992 (+ 417us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:30.601995 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:30.602004 (+ 9us) client_negotiation.cc:189] Negotiated authn=JWT
0807 18:33:30.602144 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:30.602150 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:30.603198 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:30.603202 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:30.604227 (+ 1025us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:30.604237 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:30.604761 (+ 524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:30.604765 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:30.604856 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:30.604885 (+ 29us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0807 18:33:31.070500 (+465615us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0807 18:33:31.070697 (+ 197us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.11.14.126:37181: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":142,"threads_started":1}
I20260807 18:33:31.071624 11321 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkz56XN/build/debug/bin/kudu with pid 17232
2026-08-07T18:33:31Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (765 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (132 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (136 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127579900523-11321-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:33:31 dist-test-slave-6k22 krb5kdc[17347](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:33:31 dist-test-slave-6k22 krb5kdc[17347](info): set up 2 sockets
Aug 07 18:33:31 dist-test-slave-6k22 krb5kdc[17347](info): commencing operation
krb5kdc: starting...
W20260807 18:33:32.487332 11321 mini_kdc.cc:121] Time spent starting KDC: real 1.123s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127579900523-11321-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127579900523-11321-0/krb5kdc/test-user.keytab.
Aug 07 18:33:32 dist-test-slave-6k22 krb5kdc[17347](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127612, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:33:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:33:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.14.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.14.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.14.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.126.keytab.
Entry for principal kudu/127.11.14.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.126.keytab.
I20260807 18:33:32.620718 11321 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkz56XN/build/debug/bin/kudu
/tmp/dist-test-taskkz56XN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127579900523-11321-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127579900523-11321-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127579900523-11321-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127579900523-11321-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.14.126:35541
--webserver_interface=127.11.14.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.126.keytab
--principal=kudu/127.11.14.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.14.84:45317
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.14.126:35541
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127579900523-11321-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127579900523-11321-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127579900523-11321-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:33:32.733870 17361 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:33:32.734252 17361 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:33:32.734329 17361 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:33:32.738387 17361 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:33:32.738492 17361 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:33:32.738528 17361 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:33:32.738569 17361 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:33:32.738598 17361 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:33:32.743949 17361 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.14.84:45317
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127579900523-11321-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127579900523-11321-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.14.126:35541
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.126.keytab
--principal=kudu/127.11.14.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.14.126:35541
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127579900523-11321-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.14.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17361
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127579900523-11321-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:33:32.745185 17361 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:33:32.746289 17361 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:33:32.755331 17366 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:33:32.755278 17367 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:33:32.755421 17361 server_base.cc:1034] running on GCE node
W20260807 18:33:32.755601 17369 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:33:32.756242 17361 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:33:32.757382 17361 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:33:32.758582 17361 hybrid_clock.cc:648] HybridClock initialized: now 1786127612758547 us; error 46 us; skew 500 ppm
Aug 07 18:33:32 dist-test-slave-6k22 krb5kdc[17347](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127612, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:33:32.765239 17361 init.cc:377] Logged in from keytab as kudu/127.11.14.126@KRBTEST.COM (short username kudu)
I20260807 18:33:32.766270 17361 webserver.cc:460] Webserver started at http://127.11.14.126:42401/ using document root <none> and password file <none>
I20260807 18:33:32.766878 17361 fs_manager.cc:359] Metadata directory not provided
I20260807 18:33:32.766975 17361 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:33:32.767225 17361 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:33:32.769395 17361 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127579900523-11321-0/minicluster-data/master-0/data/instance:
uuid: "f6e57a177e6c49f3bb868586143cdd80"
format_stamp: "Formatted at 2026-08-07 18:33:32 on dist-test-slave-6k22"
server_key: "19d358898e6d3d44e70a10f2faa53c75"
server_key_iv: "f8690ae063c2d2f384e5860497727e4e"
server_key_version: "encryptionkey@0"
I20260807 18:33:32.769992 17361 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127579900523-11321-0/minicluster-data/master-0/wal/instance:
uuid: "f6e57a177e6c49f3bb868586143cdd80"
format_stamp: "Formatted at 2026-08-07 18:33:32 on dist-test-slave-6k22"
server_key: "19d358898e6d3d44e70a10f2faa53c75"
server_key_iv: "f8690ae063c2d2f384e5860497727e4e"
server_key_version: "encryptionkey@0"
I20260807 18:33:32.773896 17361 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 18:33:32.776386 17376 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:33:32.777602 17361 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 18:33:32.777750 17361 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127579900523-11321-0/minicluster-data/master-0/data,/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127579900523-11321-0/minicluster-data/master-0/wal
uuid: "f6e57a177e6c49f3bb868586143cdd80"
format_stamp: "Formatted at 2026-08-07 18:33:32 on dist-test-slave-6k22"
server_key: "19d358898e6d3d44e70a10f2faa53c75"
server_key_iv: "f8690ae063c2d2f384e5860497727e4e"
server_key_version: "encryptionkey@0"
I20260807 18:33:32.777864 17361 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127579900523-11321-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127579900523-11321-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127579900523-11321-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:33:32.802695 17361 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:33:32.805995 17361 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:33:32.806200 17361 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:33:32.814002 17361 rpc_server.cc:307] RPC server started. Bound to: 127.11.14.126:35541
I20260807 18:33:32.814082 17434 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.14.126:35541 every 8 connection(s)
I20260807 18:33:32.815030 17361 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127579900523-11321-0/minicluster-data/master-0/data/info.pb
I20260807 18:33:32.817703 11321 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkz56XN/build/debug/bin/kudu as pid 17361
I20260807 18:33:32.817852 11321 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127579900523-11321-0/minicluster-data/master-0/wal/instance
I20260807 18:33:32.818146 11321 external_mini_cluster.cc:1428] Setting key 33f972a3a447176ecd203ad8d08f165f
I20260807 18:33:32.818331 17435 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:33:32.825285 17435 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f6e57a177e6c49f3bb868586143cdd80: Bootstrap starting.
Aug 07 18:33:32 dist-test-slave-6k22 krb5kdc[17347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127612, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.14.126@KRBTEST.COM
I20260807 18:33:32.828193 17435 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f6e57a177e6c49f3bb868586143cdd80: Neither blocks nor log segments found. Creating new log.
I20260807 18:33:32.829126 17435 log.cc:824] T 00000000000000000000000000000000 P f6e57a177e6c49f3bb868586143cdd80: Log is configured to *not* fsync() on all Append() calls
I20260807 18:33:32.831492 17435 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f6e57a177e6c49f3bb868586143cdd80: No bootstrap required, opened a new log
I20260807 18:33:32.833340 17438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:32.819738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35454 (local address 127.11.14.126:35541)
0807 18:33:32.820199 (+ 461us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:32.820215 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:32.820251 (+ 36us) server_negotiation.cc:407] Connection header received
0807 18:33:32.821241 (+ 990us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:32.821272 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:32.821596 (+ 324us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:32.821967 (+ 371us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:33:32.822873 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:32.823955 (+ 1082us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:32.824845 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:32.825317 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:32.825507 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:32.828430 (+ 2923us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:33:32.828435 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:33:32.828448 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:33:32.828474 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:33:32.830321 (+ 1847us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:32.830928 (+ 607us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:32.830932 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:32.830936 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:32.831043 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:32.831365 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:32.831367 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:32.831368 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:32.831828 (+ 460us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:33:32.831977 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:32.832293 (+ 316us) server_negotiation.cc:299] Negotiation successful
0807 18:33:32.832606 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":318,"thread_start_us":157,"threads_started":1}
I20260807 18:33:32.835454 17435 raft_consensus.cc:348] T 00000000000000000000000000000000 P f6e57a177e6c49f3bb868586143cdd80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6e57a177e6c49f3bb868586143cdd80" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 35541 } }
I20260807 18:33:32.835767 17435 raft_consensus.cc:374] T 00000000000000000000000000000000 P f6e57a177e6c49f3bb868586143cdd80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:33:32.835879 17435 raft_consensus.cc:729] T 00000000000000000000000000000000 P f6e57a177e6c49f3bb868586143cdd80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6e57a177e6c49f3bb868586143cdd80, State: Initialized, Role: FOLLOWER
I20260807 18:33:32.836328 17435 consensus_queue.cc:260] T 00000000000000000000000000000000 P f6e57a177e6c49f3bb868586143cdd80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6e57a177e6c49f3bb868586143cdd80" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 35541 } }
I20260807 18:33:32.836477 17435 raft_consensus.cc:388] T 00000000000000000000000000000000 P f6e57a177e6c49f3bb868586143cdd80 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:33:32.836555 17435 raft_consensus.cc:482] T 00000000000000000000000000000000 P f6e57a177e6c49f3bb868586143cdd80 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:33:32.836665 17435 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f6e57a177e6c49f3bb868586143cdd80 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:33:32.837662 17435 raft_consensus.cc:504] T 00000000000000000000000000000000 P f6e57a177e6c49f3bb868586143cdd80 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6e57a177e6c49f3bb868586143cdd80" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 35541 } }
I20260807 18:33:32.838029 17435 leader_election.cc:302] T 00000000000000000000000000000000 P f6e57a177e6c49f3bb868586143cdd80 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f6e57a177e6c49f3bb868586143cdd80; no voters:
I20260807 18:33:32.838374 17435 leader_election.cc:288] T 00000000000000000000000000000000 P f6e57a177e6c49f3bb868586143cdd80 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:33:32.838508 17440 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f6e57a177e6c49f3bb868586143cdd80 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:33:32.838755 17440 raft_consensus.cc:686] T 00000000000000000000000000000000 P f6e57a177e6c49f3bb868586143cdd80 [term 1 LEADER]: Becoming Leader. State: Replica: f6e57a177e6c49f3bb868586143cdd80, State: Running, Role: LEADER
I20260807 18:33:32.839144 17440 consensus_queue.cc:237] T 00000000000000000000000000000000 P f6e57a177e6c49f3bb868586143cdd80 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6e57a177e6c49f3bb868586143cdd80" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 35541 } }
I20260807 18:33:32.839888 17435 sys_catalog.cc:564] T 00000000000000000000000000000000 P f6e57a177e6c49f3bb868586143cdd80 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:33:32.841084 17442 sys_catalog.cc:455] T 00000000000000000000000000000000 P f6e57a177e6c49f3bb868586143cdd80 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f6e57a177e6c49f3bb868586143cdd80. Latest consensus state: current_term: 1 leader_uuid: "f6e57a177e6c49f3bb868586143cdd80" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6e57a177e6c49f3bb868586143cdd80" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 35541 } } }
I20260807 18:33:32.841214 17442 sys_catalog.cc:458] T 00000000000000000000000000000000 P f6e57a177e6c49f3bb868586143cdd80 [sys.catalog]: This master's current role is: LEADER
I20260807 18:33:32.841543 17441 sys_catalog.cc:455] T 00000000000000000000000000000000 P f6e57a177e6c49f3bb868586143cdd80 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f6e57a177e6c49f3bb868586143cdd80" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6e57a177e6c49f3bb868586143cdd80" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 35541 } } }
I20260807 18:33:32.841665 17449 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:33:32.841691 17441 sys_catalog.cc:458] T 00000000000000000000000000000000 P f6e57a177e6c49f3bb868586143cdd80 [sys.catalog]: This master's current role is: LEADER
I20260807 18:33:32.844926 17449 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:33:32.850708 17449 catalog_manager.cc:1347] Generated new cluster ID: fa00666eca8f4076bbc770b3ce19782c
I20260807 18:33:32.850770 17449 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:33:32.872741 17449 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:33:32.874153 17449 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:33:32.877769 17449 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f6e57a177e6c49f3bb868586143cdd80: Generated new TSK 0
I20260807 18:33:32.878469 17449 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.14.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.14.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.14.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.65.keytab.
Entry for principal kudu/127.11.14.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.65.keytab.
I20260807 18:33:32.915287 11321 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkz56XN/build/debug/bin/kudu
/tmp/dist-test-taskkz56XN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127579900523-11321-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127579900523-11321-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127579900523-11321-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127579900523-11321-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.14.65:0
--local_ip_for_outbound_sockets=127.11.14.65
--webserver_interface=127.11.14.65
--webserver_port=0
--tserver_master_addrs=127.11.14.126:35541
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.65.keytab
--principal=kudu/127.11.14.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.14.84:45317
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127579900523-11321-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127579900523-11321-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127579900523-11321-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:33:33.032294 17463 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:33:33.032580 17463 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:33:33.032662 17463 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:33:33.036669 17463 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:33:33.036759 17463 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:33:33.036849 17463 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.14.65
I20260807 18:33:33.042215 17463 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.14.84:45317
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127579900523-11321-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127579900523-11321-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.65.keytab
--principal=kudu/127.11.14.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.14.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127579900523-11321-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.14.65
--webserver_port=0
--tserver_master_addrs=127.11.14.126:35541
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17463
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.14.65
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127579900523-11321-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:33:33.043287 17463 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:33:33.044305 17463 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:33:33.053987 17468 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:33:33.054050 17469 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:33:33.053987 17471 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:33:33.054024 17463 server_base.cc:1034] running on GCE node
I20260807 18:33:33.054598 17463 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:33:33.055255 17463 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:33:33.056445 17463 hybrid_clock.cc:648] HybridClock initialized: now 1786127613056423 us; error 32 us; skew 500 ppm
Aug 07 18:33:33 dist-test-slave-6k22 krb5kdc[17347](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127613, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:33:33.059180 17463 init.cc:377] Logged in from keytab as kudu/127.11.14.65@KRBTEST.COM (short username kudu)
I20260807 18:33:33.060134 17463 webserver.cc:460] Webserver started at http://127.11.14.65:44147/ using document root <none> and password file <none>
I20260807 18:33:33.060798 17463 fs_manager.cc:359] Metadata directory not provided
I20260807 18:33:33.060895 17463 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:33:33.061163 17463 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:33:33.063170 17463 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127579900523-11321-0/minicluster-data/ts-0/data/instance:
uuid: "1ff0438de1d1423ab8a4979bb0f09a9e"
format_stamp: "Formatted at 2026-08-07 18:33:33 on dist-test-slave-6k22"
server_key: "410e3a9366d0180d81c05285f16cec1e"
server_key_iv: "9abee57cf3916b5da579695b309d14f3"
server_key_version: "encryptionkey@0"
I20260807 18:33:33.063822 17463 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127579900523-11321-0/minicluster-data/ts-0/wal/instance:
uuid: "1ff0438de1d1423ab8a4979bb0f09a9e"
format_stamp: "Formatted at 2026-08-07 18:33:33 on dist-test-slave-6k22"
server_key: "410e3a9366d0180d81c05285f16cec1e"
server_key_iv: "9abee57cf3916b5da579695b309d14f3"
server_key_version: "encryptionkey@0"
I20260807 18:33:33.067628 17463 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260807 18:33:33.069976 17482 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:33:33.071135 17463 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 18:33:33.071291 17463 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127579900523-11321-0/minicluster-data/ts-0/data,/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127579900523-11321-0/minicluster-data/ts-0/wal
uuid: "1ff0438de1d1423ab8a4979bb0f09a9e"
format_stamp: "Formatted at 2026-08-07 18:33:33 on dist-test-slave-6k22"
server_key: "410e3a9366d0180d81c05285f16cec1e"
server_key_iv: "9abee57cf3916b5da579695b309d14f3"
server_key_version: "encryptionkey@0"
I20260807 18:33:33.071396 17463 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127579900523-11321-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127579900523-11321-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127579900523-11321-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:33:33.083328 17463 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:33:33.086037 17463 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:33:33.086232 17463 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:33:33.086781 17463 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:33:33.087631 17463 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:33:33.087699 17463 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:33:33.087766 17463 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:33:33.087817 17463 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:33:33.097453 17463 rpc_server.cc:307] RPC server started. Bound to: 127.11.14.65:34137
I20260807 18:33:33.097482 17604 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.14.65:34137 every 8 connection(s)
I20260807 18:33:33.098376 17463 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127579900523-11321-0/minicluster-data/ts-0/data/info.pb
I20260807 18:33:33.103024 11321 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkz56XN/build/debug/bin/kudu as pid 17463
I20260807 18:33:33.103170 11321 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127579900523-11321-0/minicluster-data/ts-0/wal/instance
I20260807 18:33:33.103468 11321 external_mini_cluster.cc:1428] Setting key 6b2410b94cfa3227abea78afdb46c634
Aug 07 18:33:33 dist-test-slave-6k22 krb5kdc[17347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127613, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.65@KRBTEST.COM for kudu/127.11.14.126@KRBTEST.COM
I20260807 18:33:33.112812 17438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:33.100261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.65:49797 (local address 127.11.14.126:35541)
0807 18:33:33.100401 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:33.100405 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:33.101199 (+ 794us) server_negotiation.cc:407] Connection header received
0807 18:33:33.102588 (+ 1389us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:33.102591 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:33.102654 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:33.102789 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:33:33.104258 (+ 1469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:33.104814 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:33.105964 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:33.106199 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:33.106316 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:33.109071 (+ 2755us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:33:33.109074 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:33:33.109076 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:33:33.109099 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:33:33.110637 (+ 1538us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:33.111186 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:33.111190 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:33.111191 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:33.111283 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:33.111654 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:33.111659 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:33.111661 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:33.111854 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:33:33.111964 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:33.112503 (+ 539us) server_negotiation.cc:299] Negotiation successful
0807 18:33:33.112634 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
WARNING: no policy specified for kudu/127.11.14.66@KRBTEST.COM; defaulting to no policy
I20260807 18:33:33.113672 17607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:33.100576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:35541 (local address 127.11.14.65:49797)
0807 18:33:33.101043 (+ 467us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:33.101088 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:33.102369 (+ 1281us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:33.102997 (+ 628us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:33.103004 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:33.103476 (+ 472us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:33:33.104090 (+ 614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:33.104106 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:33.104950 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:33.104953 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:33.105824 (+ 871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:33.105842 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:33.106369 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:33.106373 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:33.106551 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:33.107160 (+ 609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:33:33.107176 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:33:33.108895 (+ 1719us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:33:33.110765 (+ 1870us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:33:33.110771 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:33:33.110786 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:33:33.111060 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:33:33.111382 (+ 322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:33:33.111386 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:33:33.111388 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:33:33.111534 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:33:33.111993 (+ 459us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:33:33.112001 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:33:33.112250 (+ 249us) client_negotiation.cc:769] Sending connection context
0807 18:33:33.112443 (+ 193us) client_negotiation.cc:240] Negotiation successful
0807 18:33:33.112773 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":278,"thread_start_us":124,"threads_started":1}
I20260807 18:33:33.114948 17605 heartbeater.cc:347] Connected to a master server at 127.11.14.126:35541
I20260807 18:33:33.115206 17605 heartbeater.cc:464] Registering TS with master...
I20260807 18:33:33.115844 17605 heartbeater.cc:511] Master 127.11.14.126:35541 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.14.66@KRBTEST.COM" created.
I20260807 18:33:33.117523 17393 ts_manager.cc:194] Registered new tserver with Master: 1ff0438de1d1423ab8a4979bb0f09a9e (127.11.14.65:34137)
I20260807 18:33:33.119057 17393 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.14.65@KRBTEST.COM'} at 127.11.14.65:49797
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.14.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.66.keytab.
Entry for principal kudu/127.11.14.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.66.keytab.
I20260807 18:33:33.132704 11321 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkz56XN/build/debug/bin/kudu
/tmp/dist-test-taskkz56XN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127579900523-11321-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127579900523-11321-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127579900523-11321-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127579900523-11321-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.14.66:0
--local_ip_for_outbound_sockets=127.11.14.66
--webserver_interface=127.11.14.66
--webserver_port=0
--tserver_master_addrs=127.11.14.126:35541
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.66.keytab
--principal=kudu/127.11.14.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.14.84:45317
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127579900523-11321-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127579900523-11321-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127579900523-11321-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:33:33.246109 17610 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:33:33.246385 17610 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:33:33.246430 17610 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:33:33.250515 17610 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:33:33.250571 17610 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:33:33.250658 17610 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.14.66
I20260807 18:33:33.255964 17610 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.14.84:45317
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127579900523-11321-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127579900523-11321-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.66.keytab
--principal=kudu/127.11.14.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.14.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127579900523-11321-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.14.66
--webserver_port=0
--tserver_master_addrs=127.11.14.126:35541
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17610
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.14.66
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127579900523-11321-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:33:33.257015 17610 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:33:33.258049 17610 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:33:33.267799 17616 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:33:33.267794 17619 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:33:33.267819 17617 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:33:33.268020 17610 server_base.cc:1034] running on GCE node
I20260807 18:33:33.268471 17610 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:33:33.269145 17610 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:33:33.270334 17610 hybrid_clock.cc:648] HybridClock initialized: now 1786127613270319 us; error 32 us; skew 500 ppm
Aug 07 18:33:33 dist-test-slave-6k22 krb5kdc[17347](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127613, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:33:33.273102 17610 init.cc:377] Logged in from keytab as kudu/127.11.14.66@KRBTEST.COM (short username kudu)
I20260807 18:33:33.273895 17610 webserver.cc:460] Webserver started at http://127.11.14.66:35215/ using document root <none> and password file <none>
I20260807 18:33:33.274497 17610 fs_manager.cc:359] Metadata directory not provided
I20260807 18:33:33.274554 17610 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:33:33.274717 17610 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:33:33.276880 17610 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127579900523-11321-0/minicluster-data/ts-1/data/instance:
uuid: "0b077ab5cd2346c7bf6165fbd1fbf1ea"
format_stamp: "Formatted at 2026-08-07 18:33:33 on dist-test-slave-6k22"
server_key: "5fb120a3804fb24ce552eac695629917"
server_key_iv: "1a8e1ea82a0629dffe29113fd7bf3195"
server_key_version: "encryptionkey@0"
I20260807 18:33:33.277386 17610 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127579900523-11321-0/minicluster-data/ts-1/wal/instance:
uuid: "0b077ab5cd2346c7bf6165fbd1fbf1ea"
format_stamp: "Formatted at 2026-08-07 18:33:33 on dist-test-slave-6k22"
server_key: "5fb120a3804fb24ce552eac695629917"
server_key_iv: "1a8e1ea82a0629dffe29113fd7bf3195"
server_key_version: "encryptionkey@0"
I20260807 18:33:33.281307 17610 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 18:33:33.283854 17626 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:33:33.285125 17610 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 18:33:33.285285 17610 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127579900523-11321-0/minicluster-data/ts-1/data,/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127579900523-11321-0/minicluster-data/ts-1/wal
uuid: "0b077ab5cd2346c7bf6165fbd1fbf1ea"
format_stamp: "Formatted at 2026-08-07 18:33:33 on dist-test-slave-6k22"
server_key: "5fb120a3804fb24ce552eac695629917"
server_key_iv: "1a8e1ea82a0629dffe29113fd7bf3195"
server_key_version: "encryptionkey@0"
I20260807 18:33:33.285393 17610 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127579900523-11321-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127579900523-11321-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127579900523-11321-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:33:33.303529 17610 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:33:33.306456 17610 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:33:33.306658 17610 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:33:33.307274 17610 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:33:33.308182 17610 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:33:33.308230 17610 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:33:33.308298 17610 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:33:33.308331 17610 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:33:33.317591 17610 rpc_server.cc:307] RPC server started. Bound to: 127.11.14.66:35301
I20260807 18:33:33.317613 17750 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.14.66:35301 every 8 connection(s)
I20260807 18:33:33.318595 17610 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127579900523-11321-0/minicluster-data/ts-1/data/info.pb
I20260807 18:33:33.319352 11321 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkz56XN/build/debug/bin/kudu as pid 17610
I20260807 18:33:33.319497 11321 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127579900523-11321-0/minicluster-data/ts-1/wal/instance
I20260807 18:33:33.319893 11321 external_mini_cluster.cc:1428] Setting key 759b0a89aa659866cf78c0ecbf48b33d
Aug 07 18:33:33 dist-test-slave-6k22 krb5kdc[17347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127613, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.66@KRBTEST.COM for kudu/127.11.14.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.11.14.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.14.67@KRBTEST.COM" created.
I20260807 18:33:33.333688 17438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:33.320905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.66:42431 (local address 127.11.14.126:35541)
0807 18:33:33.321054 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:33.321059 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:33.322026 (+ 967us) server_negotiation.cc:407] Connection header received
0807 18:33:33.323438 (+ 1412us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:33.323441 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:33.323493 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:33.323600 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:33:33.324811 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:33.325373 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:33.326535 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:33.326770 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:33.326881 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:33.329638 (+ 2757us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:33:33.329642 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:33:33.329644 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:33:33.329672 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:33:33.331345 (+ 1673us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:33.331935 (+ 590us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:33.331939 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:33.331941 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:33.332046 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:33.332460 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:33.332462 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:33.332464 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:33.332666 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:33:33.332782 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:33.333358 (+ 576us) server_negotiation.cc:299] Negotiation successful
0807 18:33:33.333496 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260807 18:33:33.334479 17753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:33.321330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:35541 (local address 127.11.14.66:42431)
0807 18:33:33.321877 (+ 547us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:33.321922 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:33.323224 (+ 1302us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:33.323785 (+ 561us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:33.323795 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:33.324169 (+ 374us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:33:33.324661 (+ 492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:33.324675 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:33.325504 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:33.325508 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:33.326395 (+ 887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:33.326404 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:33.326927 (+ 523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:33.326932 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:33.327109 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:33.327782 (+ 673us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:33:33.327799 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:33:33.329470 (+ 1671us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:33:33.331472 (+ 2002us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:33:33.331478 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:33:33.331493 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:33:33.331828 (+ 335us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:33:33.332200 (+ 372us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:33:33.332207 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:33:33.332210 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:33:33.332362 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:33:33.332785 (+ 423us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:33:33.332794 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:33:33.333106 (+ 312us) client_negotiation.cc:769] Sending connection context
0807 18:33:33.333310 (+ 204us) client_negotiation.cc:240] Negotiation successful
0807 18:33:33.333659 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":303,"thread_start_us":113,"threads_started":1}
I20260807 18:33:33.335845 17751 heartbeater.cc:347] Connected to a master server at 127.11.14.126:35541
I20260807 18:33:33.336092 17751 heartbeater.cc:464] Registering TS with master...
I20260807 18:33:33.336606 17751 heartbeater.cc:511] Master 127.11.14.126:35541 requested a full tablet report, sending...
I20260807 18:33:33.337831 17393 ts_manager.cc:194] Registered new tserver with Master: 0b077ab5cd2346c7bf6165fbd1fbf1ea (127.11.14.66:35301)
I20260807 18:33:33.338466 17393 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.14.66@KRBTEST.COM'} at 127.11.14.66:42431
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.14.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.67.keytab.
Entry for principal kudu/127.11.14.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.67.keytab.
I20260807 18:33:33.350636 11321 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkz56XN/build/debug/bin/kudu
/tmp/dist-test-taskkz56XN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127579900523-11321-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127579900523-11321-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127579900523-11321-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127579900523-11321-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.14.67:0
--local_ip_for_outbound_sockets=127.11.14.67
--webserver_interface=127.11.14.67
--webserver_port=0
--tserver_master_addrs=127.11.14.126:35541
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.67.keytab
--principal=kudu/127.11.14.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.14.84:45317
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127579900523-11321-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127579900523-11321-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127579900523-11321-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:33:33.468524 17756 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:33:33.468806 17756 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:33:33.468850 17756 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:33:33.472918 17756 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:33:33.472982 17756 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:33:33.473045 17756 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.14.67
I20260807 18:33:33.478616 17756 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.14.84:45317
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127579900523-11321-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127579900523-11321-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.67.keytab
--principal=kudu/127.11.14.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.14.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127579900523-11321-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.14.67
--webserver_port=0
--tserver_master_addrs=127.11.14.126:35541
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17756
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.14.67
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127579900523-11321-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:33:33.479790 17756 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:33:33.480880 17756 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:33:33.491081 17761 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:33:33.491101 17764 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:33:33.491082 17762 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:33:33.491333 17756 server_base.cc:1034] running on GCE node
I20260807 18:33:33.491901 17756 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:33:33.492509 17756 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:33:33.493676 17756 hybrid_clock.cc:648] HybridClock initialized: now 1786127613493661 us; error 25 us; skew 500 ppm
Aug 07 18:33:33 dist-test-slave-6k22 krb5kdc[17347](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127613, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:33:33.496627 17756 init.cc:377] Logged in from keytab as kudu/127.11.14.67@KRBTEST.COM (short username kudu)
I20260807 18:33:33.497495 17756 webserver.cc:460] Webserver started at http://127.11.14.67:42969/ using document root <none> and password file <none>
I20260807 18:33:33.498034 17756 fs_manager.cc:359] Metadata directory not provided
I20260807 18:33:33.498104 17756 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:33:33.498308 17756 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:33:33.500447 17756 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127579900523-11321-0/minicluster-data/ts-2/data/instance:
uuid: "3af85e93f9d0446fa18aa8e0877f30d9"
format_stamp: "Formatted at 2026-08-07 18:33:33 on dist-test-slave-6k22"
server_key: "5879576d3bf487669dee689519ba5e26"
server_key_iv: "7873b7b3f0c7d2505494903c0db8f6c2"
server_key_version: "encryptionkey@0"
I20260807 18:33:33.500977 17756 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127579900523-11321-0/minicluster-data/ts-2/wal/instance:
uuid: "3af85e93f9d0446fa18aa8e0877f30d9"
format_stamp: "Formatted at 2026-08-07 18:33:33 on dist-test-slave-6k22"
server_key: "5879576d3bf487669dee689519ba5e26"
server_key_iv: "7873b7b3f0c7d2505494903c0db8f6c2"
server_key_version: "encryptionkey@0"
I20260807 18:33:33.504841 17756 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.003s
I20260807 18:33:33.507392 17771 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:33:33.508665 17756 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260807 18:33:33.508806 17756 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127579900523-11321-0/minicluster-data/ts-2/data,/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127579900523-11321-0/minicluster-data/ts-2/wal
uuid: "3af85e93f9d0446fa18aa8e0877f30d9"
format_stamp: "Formatted at 2026-08-07 18:33:33 on dist-test-slave-6k22"
server_key: "5879576d3bf487669dee689519ba5e26"
server_key_iv: "7873b7b3f0c7d2505494903c0db8f6c2"
server_key_version: "encryptionkey@0"
I20260807 18:33:33.508917 17756 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127579900523-11321-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127579900523-11321-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127579900523-11321-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:33:33.521481 17756 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:33:33.524415 17756 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:33:33.524624 17756 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:33:33.525259 17756 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:33:33.526168 17756 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:33:33.526217 17756 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:33:33.526297 17756 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:33:33.526340 17756 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:33:33.535344 17756 rpc_server.cc:307] RPC server started. Bound to: 127.11.14.67:36321
I20260807 18:33:33.535365 17894 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.14.67:36321 every 8 connection(s)
I20260807 18:33:33.536454 17756 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127579900523-11321-0/minicluster-data/ts-2/data/info.pb
I20260807 18:33:33.537885 11321 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkz56XN/build/debug/bin/kudu as pid 17756
I20260807 18:33:33.538022 11321 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127579900523-11321-0/minicluster-data/ts-2/wal/instance
I20260807 18:33:33.538353 11321 external_mini_cluster.cc:1428] Setting key 72537d4711dead4cb7c442bf3390740c
Aug 07 18:33:33 dist-test-slave-6k22 krb5kdc[17347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127613, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.67@KRBTEST.COM for kudu/127.11.14.126@KRBTEST.COM
I20260807 18:33:33.551378 17438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:33.538707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.67:41325 (local address 127.11.14.126:35541)
0807 18:33:33.538923 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:33.538929 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:33.540032 (+ 1103us) server_negotiation.cc:407] Connection header received
0807 18:33:33.541790 (+ 1758us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:33.541793 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:33.541855 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:33.541944 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:33:33.543456 (+ 1512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:33.544046 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:33.545177 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:33.545388 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:33.545469 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:33.548039 (+ 2570us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:33:33.548043 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:33:33.548045 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:33:33.548068 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:33:33.549450 (+ 1382us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:33.549946 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:33.549948 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:33.549950 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:33.550024 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:33.550353 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:33.550355 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:33.550356 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:33.550527 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:33:33.550608 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:33.551108 (+ 500us) server_negotiation.cc:299] Negotiation successful
0807 18:33:33.551208 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260807 18:33:33.552531 17897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:33.539236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:35541 (local address 127.11.14.67:41325)
0807 18:33:33.539837 (+ 601us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:33.539920 (+ 83us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:33.541513 (+ 1593us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:33.542150 (+ 637us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:33.542162 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:33.542771 (+ 609us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:33:33.543303 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:33.543338 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:33.544165 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:33.544167 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:33.545055 (+ 888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:33.545065 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:33.545524 (+ 459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:33.545526 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:33.545673 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:33.546308 (+ 635us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:33:33.546325 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:33:33.547866 (+ 1541us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:33:33.549573 (+ 1707us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:33:33.549578 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:33:33.549590 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:33:33.549853 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:33:33.550125 (+ 272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:33:33.550127 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:33:33.550129 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:33:33.550265 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:33:33.550631 (+ 366us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:33:33.550638 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:33:33.550874 (+ 236us) client_negotiation.cc:769] Sending connection context
0807 18:33:33.551077 (+ 203us) client_negotiation.cc:240] Negotiation successful
0807 18:33:33.551408 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":352,"thread_start_us":193,"threads_started":1}
I20260807 18:33:33.553529 17895 heartbeater.cc:347] Connected to a master server at 127.11.14.126:35541
I20260807 18:33:33.553740 17895 heartbeater.cc:464] Registering TS with master...
I20260807 18:33:33.554199 17895 heartbeater.cc:511] Master 127.11.14.126:35541 requested a full tablet report, sending...
I20260807 18:33:33.555231 17393 ts_manager.cc:194] Registered new tserver with Master: 3af85e93f9d0446fa18aa8e0877f30d9 (127.11.14.67:36321)
I20260807 18:33:33.555892 17393 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.14.67@KRBTEST.COM'} at 127.11.14.67:41325
I20260807 18:33:33.564604 11321 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127579900523-11321-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127579900523-11321-0/krb5kdc/test-admin.keytab.
Aug 07 18:33:33 dist-test-slave-6k22 krb5kdc[17347](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127613, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:33:33.582480 11321 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260807 18:33:33.585547 17902 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:33:33.583965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56186 (local address 127.11.14.65:34137)
0807 18:33:33.584315 (+ 350us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:33.584330 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:33.584371 (+ 41us) server_negotiation.cc:407] Connection header received
0807 18:33:33.584425 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:33.584429 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:33.584626 (+ 197us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:33.584765 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:33:33.585329 (+ 564us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56186: BlockingRecv error: recv got EOF from 127.0.0.1:56186 (error 108)
Metrics: {"server-negotiator.queue_time_us":242,"thread_start_us":146,"threads_started":1}
W20260807 18:33:33.585654 11321 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127579900523-11321-0/krb5kdc/krb5cc' of type FILE
Aug 07 18:33:33 dist-test-slave-6k22 krb5kdc[17347](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127613, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:33:33.586719 11321 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 07 18:33:33 dist-test-slave-6k22 krb5kdc[17347](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127613, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:33:33.587707 11321 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 07 18:33:33 dist-test-slave-6k22 krb5kdc[17347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127613, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.14.65@KRBTEST.COM
I20260807 18:33:33.597703 17902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:33.588545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56190 (local address 127.11.14.65:34137)
0807 18:33:33.588676 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:33.588680 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:33.588885 (+ 205us) server_negotiation.cc:407] Connection header received
0807 18:33:33.589036 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:33.589039 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:33.589086 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:33.589169 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:33:33.589703 (+ 534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:33.590560 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:33.591410 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:33.591852 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:33.591962 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:33.594241 (+ 2279us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:33:33.594251 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:33:33.594264 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:33:33.594286 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:33:33.595900 (+ 1614us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:33.596351 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:33.596356 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:33.596357 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:33.596434 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:33.596719 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:33.596723 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:33.596725 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:33.596966 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:33:33.597206 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:33.597400 (+ 194us) server_negotiation.cc:299] Negotiation successful
0807 18:33:33.597574 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
W20260807 18:33:33.601661 11321 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127579900523-11321-0/krb5kdc/krb5cc' of type FILE
W20260807 18:33:33.601712 17902 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:33:33.600708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56198 (local address 127.11.14.65:34137)
0807 18:33:33.600844 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:33.600848 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:33.600861 (+ 13us) server_negotiation.cc:407] Connection header received
0807 18:33:33.600958 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:33.600961 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:33.600996 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:33.601085 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:33:33.601525 (+ 440us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56198: BlockingRecv error: recv got EOF from 127.0.0.1:56198 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Aug 07 18:33:33 dist-test-slave-6k22 krb5kdc[17347](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127613, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:33:33.602779 11321 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 07 18:33:33 dist-test-slave-6k22 krb5kdc[17347](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127613, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:33:33.603819 11321 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 07 18:33:33 dist-test-slave-6k22 krb5kdc[17347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127613, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.14.65@KRBTEST.COM
I20260807 18:33:33.612579 17902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:33.604689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56206 (local address 127.11.14.65:34137)
0807 18:33:33.604815 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:33.604820 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:33.605008 (+ 188us) server_negotiation.cc:407] Connection header received
0807 18:33:33.605128 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:33.605132 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:33.605186 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:33.605272 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:33:33.605775 (+ 503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:33.606314 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:33.607168 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:33.607385 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:33.607495 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:33.609703 (+ 2208us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:33:33.609706 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:33:33.609708 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:33:33.609729 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:33:33.610956 (+ 1227us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:33.611349 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:33.611352 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:33.611353 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:33.611423 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:33.611844 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:33.611847 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:33.611848 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:33.612006 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:33:33.612106 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:33.612317 (+ 211us) server_negotiation.cc:299] Negotiation successful
0807 18:33:33.612408 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
W20260807 18:33:33.615352 17902 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:33:33.614342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56222 (local address 127.11.14.65:34137)
0807 18:33:33.614454 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:33.614458 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:33.614615 (+ 157us) server_negotiation.cc:407] Connection header received
0807 18:33:33.614750 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:33.614758 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:33.614791 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:33.614873 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:33:33.615258 (+ 385us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56222: BlockingRecv error: recv got EOF from 127.0.0.1:56222 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
W20260807 18:33:33.615711 11321 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786127579900523-11321-0/krb5kdc/krb5cc' of type FILE
Aug 07 18:33:33 dist-test-slave-6k22 krb5kdc[17347](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127613, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:33:33.616828 11321 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 07 18:33:33 dist-test-slave-6k22 krb5kdc[17347](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127613, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:33:33.617854 11321 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 07 18:33:33 dist-test-slave-6k22 krb5kdc[17347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127613, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.14.65@KRBTEST.COM
I20260807 18:33:33.626986 17902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:33.618797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56236 (local address 127.11.14.65:34137)
0807 18:33:33.618944 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:33.618951 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:33.618971 (+ 20us) server_negotiation.cc:407] Connection header received
0807 18:33:33.619139 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:33.619147 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:33.619184 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:33.619273 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:33:33.619937 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:33.620429 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:33.621257 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:33.621473 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:33.621593 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:33.623920 (+ 2327us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:33:33.623923 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:33:33.623925 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:33:33.623946 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:33:33.625289 (+ 1343us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:33.625771 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:33.625777 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:33.625779 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:33.625864 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:33.626149 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:33.626153 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:33.626155 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:33.626325 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:33:33.626461 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:33.626726 (+ 265us) server_negotiation.cc:299] Negotiation successful
0807 18:33:33.626832 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260807 18:33:33.628088 11321 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkz56XN/build/debug/bin/kudu with pid 17463
I20260807 18:33:33.636888 11321 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkz56XN/build/debug/bin/kudu with pid 17610
I20260807 18:33:33.645594 11321 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkz56XN/build/debug/bin/kudu with pid 17756
I20260807 18:33:33.657322 11321 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkz56XN/build/debug/bin/kudu with pid 17361
2026-08-07T18:33:33Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (2317 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:33:33 dist-test-slave-6k22 krb5kdc[17915](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:33:33 dist-test-slave-6k22 krb5kdc[17915](info): set up 2 sockets
Aug 07 18:33:33 dist-test-slave-6k22 krb5kdc[17915](info): commencing operation
krb5kdc: starting...
W20260807 18:33:34.763876 11321 mini_kdc.cc:121] Time spent starting KDC: real 1.082s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/krb5kdc/test-user.keytab.
Aug 07 18:33:34 dist-test-slave-6k22 krb5kdc[17915](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127614, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:33:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:33:34Z Disabled control of system clock
WARNING: no policy specified for oryx/127.11.14.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.14.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.14.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/krb5kdc/oryx_127.11.14.126.keytab.
Entry for principal oryx/127.11.14.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/krb5kdc/oryx_127.11.14.126.keytab.
I20260807 18:33:34.899169 11321 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkz56XN/build/debug/bin/kudu
/tmp/dist-test-taskkz56XN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.14.126:45409
--webserver_interface=127.11.14.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/krb5kdc/oryx_127.11.14.126.keytab
--principal=oryx/127.11.14.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.14.84:41437
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.14.126:45409
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:33:35.013190 17928 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:33:35.013468 17928 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:33:35.013509 17928 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:33:35.017477 17928 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:33:35.017529 17928 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:33:35.017549 17928 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:33:35.017566 17928 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:33:35.017581 17928 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:33:35.022903 17928 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.14.84:41437
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.14.126:45409
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/krb5kdc/oryx_127.11.14.126.keytab
--principal=oryx/127.11.14.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.14.126:45409
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.14.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17928
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:33:35.023970 17928 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:33:35.024997 17928 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:33:35.033710 17935 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:33:35.033769 17937 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:33:35.033776 17934 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:33:35.033823 17928 server_base.cc:1034] running on GCE node
I20260807 18:33:35.034546 17928 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:33:35.035867 17928 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:33:35.037047 17928 hybrid_clock.cc:648] HybridClock initialized: now 1786127615037018 us; error 39 us; skew 500 ppm
Aug 07 18:33:35 dist-test-slave-6k22 krb5kdc[17915](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127615, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.14.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:33:35.040200 17928 init.cc:377] Logged in from keytab as oryx/127.11.14.126@KRBTEST.COM (short username oryx)
I20260807 18:33:35.041227 17928 webserver.cc:460] Webserver started at http://127.11.14.126:35091/ using document root <none> and password file <none>
I20260807 18:33:35.041865 17928 fs_manager.cc:359] Metadata directory not provided
I20260807 18:33:35.041989 17928 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:33:35.042285 17928 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:33:35.044667 17928 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/minicluster-data/master-0/data/instance:
uuid: "9b176ea138c9402a95ef84541f6cc5e3"
format_stamp: "Formatted at 2026-08-07 18:33:35 on dist-test-slave-6k22"
server_key: "0fe018c6bd3ff21d7d5eaf5449f74492"
server_key_iv: "7eef299ba559dc056c0ce383c424bf82"
server_key_version: "encryptionkey@0"
I20260807 18:33:35.045372 17928 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/minicluster-data/master-0/wal/instance:
uuid: "9b176ea138c9402a95ef84541f6cc5e3"
format_stamp: "Formatted at 2026-08-07 18:33:35 on dist-test-slave-6k22"
server_key: "0fe018c6bd3ff21d7d5eaf5449f74492"
server_key_iv: "7eef299ba559dc056c0ce383c424bf82"
server_key_version: "encryptionkey@0"
I20260807 18:33:35.049654 17928 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 18:33:35.052345 17944 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:33:35.053658 17928 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.001s
I20260807 18:33:35.053797 17928 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/minicluster-data/master-0/data,/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/minicluster-data/master-0/wal
uuid: "9b176ea138c9402a95ef84541f6cc5e3"
format_stamp: "Formatted at 2026-08-07 18:33:35 on dist-test-slave-6k22"
server_key: "0fe018c6bd3ff21d7d5eaf5449f74492"
server_key_iv: "7eef299ba559dc056c0ce383c424bf82"
server_key_version: "encryptionkey@0"
I20260807 18:33:35.053905 17928 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:33:35.074220 17928 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:33:35.077198 17928 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:33:35.077417 17928 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:33:35.084609 17928 rpc_server.cc:307] RPC server started. Bound to: 127.11.14.126:45409
I20260807 18:33:35.084908 18000 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.14.126:45409 every 8 connection(s)
I20260807 18:33:35.085630 17928 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/minicluster-data/master-0/data/info.pb
I20260807 18:33:35.088424 18001 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:33:35.094053 18001 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9b176ea138c9402a95ef84541f6cc5e3: Bootstrap starting.
I20260807 18:33:35.095801 11321 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkz56XN/build/debug/bin/kudu as pid 17928
I20260807 18:33:35.095985 11321 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/minicluster-data/master-0/wal/instance
I20260807 18:33:35.096335 11321 external_mini_cluster.cc:1428] Setting key 25ca32ec9715d8375774857e63dd6eb8
I20260807 18:33:35.096722 18001 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9b176ea138c9402a95ef84541f6cc5e3: Neither blocks nor log segments found. Creating new log.
I20260807 18:33:35.097649 18001 log.cc:824] T 00000000000000000000000000000000 P 9b176ea138c9402a95ef84541f6cc5e3: Log is configured to *not* fsync() on all Append() calls
I20260807 18:33:35.099771 18001 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9b176ea138c9402a95ef84541f6cc5e3: No bootstrap required, opened a new log
I20260807 18:33:35.103765 18001 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9b176ea138c9402a95ef84541f6cc5e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b176ea138c9402a95ef84541f6cc5e3" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 45409 } }
I20260807 18:33:35.104054 18001 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9b176ea138c9402a95ef84541f6cc5e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:33:35.104133 18001 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9b176ea138c9402a95ef84541f6cc5e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b176ea138c9402a95ef84541f6cc5e3, State: Initialized, Role: FOLLOWER
I20260807 18:33:35.104676 18001 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9b176ea138c9402a95ef84541f6cc5e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b176ea138c9402a95ef84541f6cc5e3" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 45409 } }
I20260807 18:33:35.104877 18001 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9b176ea138c9402a95ef84541f6cc5e3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:33:35.104951 18001 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9b176ea138c9402a95ef84541f6cc5e3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:33:35.105073 18001 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9b176ea138c9402a95ef84541f6cc5e3 [term 0 FOLLOWER]: Advancing to term 1
Aug 07 18:33:35 dist-test-slave-6k22 krb5kdc[17915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127614, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.11.14.126@KRBTEST.COM
I20260807 18:33:35.106042 18001 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9b176ea138c9402a95ef84541f6cc5e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b176ea138c9402a95ef84541f6cc5e3" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 45409 } }
I20260807 18:33:35.106446 18001 leader_election.cc:302] T 00000000000000000000000000000000 P 9b176ea138c9402a95ef84541f6cc5e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9b176ea138c9402a95ef84541f6cc5e3; no voters:
I20260807 18:33:35.106770 18001 leader_election.cc:288] T 00000000000000000000000000000000 P 9b176ea138c9402a95ef84541f6cc5e3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:33:35.106885 18006 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9b176ea138c9402a95ef84541f6cc5e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:33:35.107435 18006 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9b176ea138c9402a95ef84541f6cc5e3 [term 1 LEADER]: Becoming Leader. State: Replica: 9b176ea138c9402a95ef84541f6cc5e3, State: Running, Role: LEADER
I20260807 18:33:35.107846 18006 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9b176ea138c9402a95ef84541f6cc5e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b176ea138c9402a95ef84541f6cc5e3" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 45409 } }
I20260807 18:33:35.108228 18001 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9b176ea138c9402a95ef84541f6cc5e3 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:33:35.109553 18007 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b176ea138c9402a95ef84541f6cc5e3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9b176ea138c9402a95ef84541f6cc5e3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b176ea138c9402a95ef84541f6cc5e3" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 45409 } } }
I20260807 18:33:35.109714 18007 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b176ea138c9402a95ef84541f6cc5e3 [sys.catalog]: This master's current role is: LEADER
I20260807 18:33:35.110103 18008 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b176ea138c9402a95ef84541f6cc5e3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9b176ea138c9402a95ef84541f6cc5e3. Latest consensus state: current_term: 1 leader_uuid: "9b176ea138c9402a95ef84541f6cc5e3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b176ea138c9402a95ef84541f6cc5e3" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 45409 } } }
I20260807 18:33:35.110246 18008 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b176ea138c9402a95ef84541f6cc5e3 [sys.catalog]: This master's current role is: LEADER
I20260807 18:33:35.112744 18004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:35.097560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32936 (local address 127.11.14.126:45409)
0807 18:33:35.098016 (+ 456us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:35.098033 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:35.098101 (+ 68us) server_negotiation.cc:407] Connection header received
0807 18:33:35.098966 (+ 865us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:35.098987 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:35.099327 (+ 340us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:35.099712 (+ 385us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:33:35.100559 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:35.101712 (+ 1153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:35.103001 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:35.103464 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:35.103732 (+ 268us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:35.106687 (+ 2955us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:33:35.106698 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:33:35.106719 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:33:35.106753 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:33:35.109004 (+ 2251us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:35.109608 (+ 604us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:35.109615 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:35.109621 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:35.109756 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:35.110123 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:35.110126 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:35.110128 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:35.110704 (+ 576us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:33:35.110941 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:35.111420 (+ 479us) server_negotiation.cc:299] Negotiation successful
0807 18:33:35.111770 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":297,"thread_start_us":152,"threads_started":1}
W20260807 18:33:35.113778 18022 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9b176ea138c9402a95ef84541f6cc5e3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 18:33:35.113906 18022 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 18:33:35.114012 18015 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:33:35.114713 18015 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:33:35.120765 18015 catalog_manager.cc:1347] Generated new cluster ID: b84b754b281a4b06b86e0ce85d0fbbe8
I20260807 18:33:35.120833 18015 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:33:35.133760 18015 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:33:35.134611 18015 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:33:35.138767 18015 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9b176ea138c9402a95ef84541f6cc5e3: Generated new TSK 0
I20260807 18:33:35.139526 18015 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.11.14.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.14.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.14.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/krb5kdc/oryx_127.11.14.65.keytab.
Entry for principal oryx/127.11.14.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/krb5kdc/oryx_127.11.14.65.keytab.
I20260807 18:33:35.175299 11321 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkz56XN/build/debug/bin/kudu
/tmp/dist-test-taskkz56XN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.14.65:0
--local_ip_for_outbound_sockets=127.11.14.65
--webserver_interface=127.11.14.65
--webserver_port=0
--tserver_master_addrs=127.11.14.126:45409
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/krb5kdc/oryx_127.11.14.65.keytab
--principal=oryx/127.11.14.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.14.84:41437
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:33:35.292732 18029 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:33:35.292986 18029 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:33:35.293028 18029 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:33:35.296995 18029 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:33:35.297048 18029 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:33:35.297106 18029 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.14.65
I20260807 18:33:35.302233 18029 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.14.84:41437
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/krb5kdc/oryx_127.11.14.65.keytab
--principal=oryx/127.11.14.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.14.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.14.65
--webserver_port=0
--tserver_master_addrs=127.11.14.126:45409
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18029
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.14.65
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:33:35.303188 18029 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:33:35.304153 18029 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:33:35.313997 18038 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:33:35.313978 18029 server_base.cc:1034] running on GCE node
W20260807 18:33:35.313946 18035 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:33:35.313967 18036 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:33:35.314644 18029 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:33:35.315330 18029 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:33:35.316506 18029 hybrid_clock.cc:648] HybridClock initialized: now 1786127615316457 us; error 70 us; skew 500 ppm
Aug 07 18:33:35 dist-test-slave-6k22 krb5kdc[17915](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127615, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.14.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:33:35.319214 18029 init.cc:377] Logged in from keytab as oryx/127.11.14.65@KRBTEST.COM (short username oryx)
I20260807 18:33:35.320286 18029 webserver.cc:460] Webserver started at http://127.11.14.65:38645/ using document root <none> and password file <none>
I20260807 18:33:35.320876 18029 fs_manager.cc:359] Metadata directory not provided
I20260807 18:33:35.320972 18029 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:33:35.321208 18029 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:33:35.323289 18029 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/minicluster-data/ts-0/data/instance:
uuid: "a290fe9b958d48f0a540e728e58c4bc7"
format_stamp: "Formatted at 2026-08-07 18:33:35 on dist-test-slave-6k22"
server_key: "005363606f383b886e3255a05d63e888"
server_key_iv: "9e4faa765082d77eb388fab665733aca"
server_key_version: "encryptionkey@0"
I20260807 18:33:35.323922 18029 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/minicluster-data/ts-0/wal/instance:
uuid: "a290fe9b958d48f0a540e728e58c4bc7"
format_stamp: "Formatted at 2026-08-07 18:33:35 on dist-test-slave-6k22"
server_key: "005363606f383b886e3255a05d63e888"
server_key_iv: "9e4faa765082d77eb388fab665733aca"
server_key_version: "encryptionkey@0"
I20260807 18:33:35.327878 18029 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260807 18:33:35.330376 18047 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:33:35.331674 18029 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 18:33:35.331813 18029 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/minicluster-data/ts-0/data,/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/minicluster-data/ts-0/wal
uuid: "a290fe9b958d48f0a540e728e58c4bc7"
format_stamp: "Formatted at 2026-08-07 18:33:35 on dist-test-slave-6k22"
server_key: "005363606f383b886e3255a05d63e888"
server_key_iv: "9e4faa765082d77eb388fab665733aca"
server_key_version: "encryptionkey@0"
I20260807 18:33:35.331934 18029 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:33:35.342314 18029 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:33:35.345322 18029 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:33:35.345531 18029 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:33:35.346117 18029 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:33:35.346947 18029 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:33:35.347021 18029 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:33:35.347092 18029 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:33:35.347137 18029 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:33:35.356148 18029 rpc_server.cc:307] RPC server started. Bound to: 127.11.14.65:33879
I20260807 18:33:35.356272 18162 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.14.65:33879 every 8 connection(s)
I20260807 18:33:35.357117 18029 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/minicluster-data/ts-0/data/info.pb
I20260807 18:33:35.362171 11321 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkz56XN/build/debug/bin/kudu as pid 18029
I20260807 18:33:35.362334 11321 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/minicluster-data/ts-0/wal/instance
I20260807 18:33:35.362646 11321 external_mini_cluster.cc:1428] Setting key 2a79494a451211a244187f8a7749c2a2
Aug 07 18:33:35 dist-test-slave-6k22 krb5kdc[17915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127615, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.14.65@KRBTEST.COM for oryx/127.11.14.126@KRBTEST.COM
I20260807 18:33:35.372445 18004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:35.359122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.65:44073 (local address 127.11.14.126:45409)
0807 18:33:35.359305 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:35.359310 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:35.360426 (+ 1116us) server_negotiation.cc:407] Connection header received
0807 18:33:35.361278 (+ 852us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:35.361281 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:35.361329 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:35.361406 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:33:35.362709 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:35.363444 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:35.364952 (+ 1508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:35.365365 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:35.365496 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:35.368579 (+ 3083us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:33:35.368582 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:33:35.368584 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:33:35.368607 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:33:35.370094 (+ 1487us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:35.370748 (+ 654us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:35.370751 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:35.370752 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:35.370829 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:35.371260 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:35.371262 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:35.371263 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:35.371437 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:33:35.371510 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:35.372200 (+ 690us) server_negotiation.cc:299] Negotiation successful
0807 18:33:35.372309 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260807 18:33:35.373698 18165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:35.359539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:45409 (local address 127.11.14.65:44073)
0807 18:33:35.360261 (+ 722us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:35.360302 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:35.361076 (+ 774us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:35.361583 (+ 507us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:35.361589 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:35.361947 (+ 358us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:33:35.362545 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:35.362562 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:35.363616 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:35.363620 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:35.364812 (+ 1192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:35.364824 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:35.365515 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:35.365518 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:35.365674 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:35.366323 (+ 649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:33:35.366361 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:33:35.368403 (+ 2042us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:33:35.370224 (+ 1821us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:33:35.370233 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:33:35.370251 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:33:35.370637 (+ 386us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:33:35.370927 (+ 290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:33:35.370933 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:33:35.370937 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:33:35.371143 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:33:35.371570 (+ 427us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:33:35.371578 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:33:35.371919 (+ 341us) client_negotiation.cc:769] Sending connection context
0807 18:33:35.372145 (+ 226us) client_negotiation.cc:240] Negotiation successful
0807 18:33:35.372521 (+ 376us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":471,"thread_start_us":116,"threads_started":1}
WARNING: no policy specified for oryx/127.11.14.66@KRBTEST.COM; defaulting to no policy
I20260807 18:33:35.374979 18163 heartbeater.cc:347] Connected to a master server at 127.11.14.126:45409
I20260807 18:33:35.375205 18163 heartbeater.cc:464] Registering TS with master...
I20260807 18:33:35.375782 18163 heartbeater.cc:511] Master 127.11.14.126:45409 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.14.66@KRBTEST.COM" created.
I20260807 18:33:35.377290 17963 ts_manager.cc:194] Registered new tserver with Master: a290fe9b958d48f0a540e728e58c4bc7 (127.11.14.65:33879)
I20260807 18:33:35.378489 17963 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.11.14.65@KRBTEST.COM'} at 127.11.14.65:44073
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.14.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/krb5kdc/oryx_127.11.14.66.keytab.
Entry for principal oryx/127.11.14.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/krb5kdc/oryx_127.11.14.66.keytab.
I20260807 18:33:35.395255 11321 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkz56XN/build/debug/bin/kudu
/tmp/dist-test-taskkz56XN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.14.66:0
--local_ip_for_outbound_sockets=127.11.14.66
--webserver_interface=127.11.14.66
--webserver_port=0
--tserver_master_addrs=127.11.14.126:45409
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/krb5kdc/oryx_127.11.14.66.keytab
--principal=oryx/127.11.14.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.14.84:41437
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:33:35.510639 18170 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:33:35.510891 18170 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:33:35.510932 18170 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:33:35.514956 18170 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:33:35.515017 18170 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:33:35.515098 18170 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.14.66
I20260807 18:33:35.520345 18170 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.14.84:41437
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/krb5kdc/oryx_127.11.14.66.keytab
--principal=oryx/127.11.14.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.14.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.14.66
--webserver_port=0
--tserver_master_addrs=127.11.14.126:45409
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18170
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.14.66
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:33:35.521450 18170 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:33:35.522423 18170 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:33:35.532155 18175 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:33:35.532150 18176 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:33:35.532112 18178 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:33:35.532135 18170 server_base.cc:1034] running on GCE node
I20260807 18:33:35.532788 18170 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:33:35.533495 18170 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:33:35.534685 18170 hybrid_clock.cc:648] HybridClock initialized: now 1786127615534672 us; error 29 us; skew 500 ppm
Aug 07 18:33:35 dist-test-slave-6k22 krb5kdc[17915](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127615, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.14.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:33:35.537532 18170 init.cc:377] Logged in from keytab as oryx/127.11.14.66@KRBTEST.COM (short username oryx)
I20260807 18:33:35.538465 18170 webserver.cc:460] Webserver started at http://127.11.14.66:35379/ using document root <none> and password file <none>
I20260807 18:33:35.539047 18170 fs_manager.cc:359] Metadata directory not provided
I20260807 18:33:35.539147 18170 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:33:35.539400 18170 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:33:35.541522 18170 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/minicluster-data/ts-1/data/instance:
uuid: "f25e947f8c2c4fef895cfd78048211d7"
format_stamp: "Formatted at 2026-08-07 18:33:35 on dist-test-slave-6k22"
server_key: "7eddcc89955fb4bed7d540a39930cd33"
server_key_iv: "dea686619bd9834f8f96201649be624f"
server_key_version: "encryptionkey@0"
I20260807 18:33:35.542126 18170 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/minicluster-data/ts-1/wal/instance:
uuid: "f25e947f8c2c4fef895cfd78048211d7"
format_stamp: "Formatted at 2026-08-07 18:33:35 on dist-test-slave-6k22"
server_key: "7eddcc89955fb4bed7d540a39930cd33"
server_key_iv: "dea686619bd9834f8f96201649be624f"
server_key_version: "encryptionkey@0"
I20260807 18:33:35.545992 18170 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 18:33:35.548467 18185 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:33:35.549649 18170 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 18:33:35.549790 18170 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/minicluster-data/ts-1/data,/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/minicluster-data/ts-1/wal
uuid: "f25e947f8c2c4fef895cfd78048211d7"
format_stamp: "Formatted at 2026-08-07 18:33:35 on dist-test-slave-6k22"
server_key: "7eddcc89955fb4bed7d540a39930cd33"
server_key_iv: "dea686619bd9834f8f96201649be624f"
server_key_version: "encryptionkey@0"
I20260807 18:33:35.549907 18170 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:33:35.569895 18170 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:33:35.573442 18170 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:33:35.573674 18170 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:33:35.574303 18170 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:33:35.575172 18170 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:33:35.575248 18170 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:33:35.575320 18170 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:33:35.575371 18170 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:33:35.585292 18170 rpc_server.cc:307] RPC server started. Bound to: 127.11.14.66:39193
I20260807 18:33:35.585320 18305 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.14.66:39193 every 8 connection(s)
I20260807 18:33:35.586217 18170 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/minicluster-data/ts-1/data/info.pb
I20260807 18:33:35.592486 11321 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkz56XN/build/debug/bin/kudu as pid 18170
I20260807 18:33:35.592628 11321 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/minicluster-data/ts-1/wal/instance
I20260807 18:33:35.592916 11321 external_mini_cluster.cc:1428] Setting key 54f7e6a3bf759e94fdff6a89b31ae719
Aug 07 18:33:35 dist-test-slave-6k22 krb5kdc[17915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127615, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.14.66@KRBTEST.COM for oryx/127.11.14.126@KRBTEST.COM
I20260807 18:33:35.602849 18004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:35.588563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.66:49799 (local address 127.11.14.126:45409)
0807 18:33:35.588745 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:35.588750 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:35.589679 (+ 929us) server_negotiation.cc:407] Connection header received
0807 18:33:35.591143 (+ 1464us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:35.591146 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:35.591193 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:35.591278 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:33:35.592676 (+ 1398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:35.593362 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:35.594908 (+ 1546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:35.595167 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:35.595293 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:35.598107 (+ 2814us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:33:35.598114 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:33:35.598120 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:33:35.598152 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:33:35.600211 (+ 2059us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:35.600978 (+ 767us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:35.600981 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:35.600982 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:35.601062 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:35.601546 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:35.601548 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:35.601549 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:35.601748 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:33:35.601828 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:35.602601 (+ 773us) server_negotiation.cc:299] Negotiation successful
0807 18:33:35.602700 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260807 18:33:35.604243 18308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:35.589020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:45409 (local address 127.11.14.66:49799)
0807 18:33:35.589502 (+ 482us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:35.589548 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:35.590928 (+ 1380us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:35.591468 (+ 540us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:35.591476 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:35.591974 (+ 498us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:33:35.592503 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:35.592518 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:35.593510 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:35.593519 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:35.594765 (+ 1246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:35.594776 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:35.595334 (+ 558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:35.595339 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:35.595526 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:35.596248 (+ 722us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:33:35.596265 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:33:35.597924 (+ 1659us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:33:35.600355 (+ 2431us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:33:35.600361 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:33:35.600373 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:33:35.600854 (+ 481us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:33:35.601184 (+ 330us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:33:35.601190 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:33:35.601194 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:33:35.601429 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:33:35.601877 (+ 448us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:33:35.601889 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:33:35.602293 (+ 404us) client_negotiation.cc:769] Sending connection context
0807 18:33:35.602567 (+ 274us) client_negotiation.cc:240] Negotiation successful
0807 18:33:35.602980 (+ 413us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":278,"thread_start_us":110,"threads_started":1}
WARNING: no policy specified for oryx/127.11.14.67@KRBTEST.COM; defaulting to no policy
I20260807 18:33:35.605636 18306 heartbeater.cc:347] Connected to a master server at 127.11.14.126:45409
I20260807 18:33:35.605930 18306 heartbeater.cc:464] Registering TS with master...
I20260807 18:33:35.606503 18306 heartbeater.cc:511] Master 127.11.14.126:45409 requested a full tablet report, sending...
I20260807 18:33:35.607753 17963 ts_manager.cc:194] Registered new tserver with Master: f25e947f8c2c4fef895cfd78048211d7 (127.11.14.66:39193)
I20260807 18:33:35.608371 17963 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.11.14.66@KRBTEST.COM'} at 127.11.14.66:49799
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.14.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.14.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/krb5kdc/oryx_127.11.14.67.keytab.
Entry for principal oryx/127.11.14.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/krb5kdc/oryx_127.11.14.67.keytab.
I20260807 18:33:35.626576 11321 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkz56XN/build/debug/bin/kudu
/tmp/dist-test-taskkz56XN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.14.67:0
--local_ip_for_outbound_sockets=127.11.14.67
--webserver_interface=127.11.14.67
--webserver_port=0
--tserver_master_addrs=127.11.14.126:45409
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/krb5kdc/oryx_127.11.14.67.keytab
--principal=oryx/127.11.14.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.14.84:41437
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:33:35.742511 18311 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:33:35.742795 18311 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:33:35.742870 18311 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:33:35.746994 18311 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:33:35.747073 18311 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:33:35.747190 18311 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.14.67
I20260807 18:33:35.752638 18311 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.14.84:41437
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/krb5kdc/oryx_127.11.14.67.keytab
--principal=oryx/127.11.14.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.14.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.14.67
--webserver_port=0
--tserver_master_addrs=127.11.14.126:45409
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18311
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.14.67
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:33:35.753731 18311 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:33:35.754866 18311 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:33:35.764986 18318 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:33:35.764989 18321 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:33:35.765045 18319 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:33:35.765349 18311 server_base.cc:1034] running on GCE node
I20260807 18:33:35.765739 18311 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:33:35.766402 18311 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:33:35.767625 18311 hybrid_clock.cc:648] HybridClock initialized: now 1786127615767610 us; error 26 us; skew 500 ppm
Aug 07 18:33:35 dist-test-slave-6k22 krb5kdc[17915](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127615, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.14.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:33:35.770440 18311 init.cc:377] Logged in from keytab as oryx/127.11.14.67@KRBTEST.COM (short username oryx)
I20260807 18:33:35.771335 18311 webserver.cc:460] Webserver started at http://127.11.14.67:40269/ using document root <none> and password file <none>
I20260807 18:33:35.771955 18311 fs_manager.cc:359] Metadata directory not provided
I20260807 18:33:35.772019 18311 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:33:35.772193 18311 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:33:35.774194 18311 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/minicluster-data/ts-2/data/instance:
uuid: "6a3774b03adf44d3b1475982c04da6aa"
format_stamp: "Formatted at 2026-08-07 18:33:35 on dist-test-slave-6k22"
server_key: "695e881c8d55f5d75f1119a7b5ac9b7d"
server_key_iv: "59e13a5b0d5cda5d1fe52204662cac99"
server_key_version: "encryptionkey@0"
I20260807 18:33:35.774715 18311 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/minicluster-data/ts-2/wal/instance:
uuid: "6a3774b03adf44d3b1475982c04da6aa"
format_stamp: "Formatted at 2026-08-07 18:33:35 on dist-test-slave-6k22"
server_key: "695e881c8d55f5d75f1119a7b5ac9b7d"
server_key_iv: "59e13a5b0d5cda5d1fe52204662cac99"
server_key_version: "encryptionkey@0"
I20260807 18:33:35.778469 18311 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260807 18:33:35.780911 18328 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:33:35.782189 18311 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 18:33:35.782315 18311 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/minicluster-data/ts-2/data,/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/minicluster-data/ts-2/wal
uuid: "6a3774b03adf44d3b1475982c04da6aa"
format_stamp: "Formatted at 2026-08-07 18:33:35 on dist-test-slave-6k22"
server_key: "695e881c8d55f5d75f1119a7b5ac9b7d"
server_key_iv: "59e13a5b0d5cda5d1fe52204662cac99"
server_key_version: "encryptionkey@0"
I20260807 18:33:35.782418 18311 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:33:35.794728 18311 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:33:35.798331 18311 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:33:35.798532 18311 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:33:35.799235 18311 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:33:35.800145 18311 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:33:35.800195 18311 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:33:35.800264 18311 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:33:35.800304 18311 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:33:35.809687 18311 rpc_server.cc:307] RPC server started. Bound to: 127.11.14.67:42159
I20260807 18:33:35.809755 18445 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.14.67:42159 every 8 connection(s)
I20260807 18:33:35.810701 18311 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/minicluster-data/ts-2/data/info.pb
I20260807 18:33:35.813090 11321 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkz56XN/build/debug/bin/kudu as pid 18311
I20260807 18:33:35.813220 11321 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/minicluster-data/ts-2/wal/instance
I20260807 18:33:35.813571 11321 external_mini_cluster.cc:1428] Setting key 4374a236a77fdffd753b338d9f86b157
Aug 07 18:33:35 dist-test-slave-6k22 krb5kdc[17915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127615, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.14.67@KRBTEST.COM for oryx/127.11.14.126@KRBTEST.COM
I20260807 18:33:35.824894 18004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:35.812572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.67:41201 (local address 127.11.14.126:45409)
0807 18:33:35.812716 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:35.812720 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:35.813755 (+ 1035us) server_negotiation.cc:407] Connection header received
0807 18:33:35.815508 (+ 1753us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:35.815512 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:35.815599 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:35.815688 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:33:35.816942 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:35.817458 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:35.818588 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:35.818807 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:35.818884 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:35.821492 (+ 2608us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:33:35.821501 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:33:35.821503 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:33:35.821526 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:33:35.822724 (+ 1198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:35.823257 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:35.823262 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:35.823264 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:35.823348 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:35.823775 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:35.823786 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:35.823788 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:35.823967 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:33:35.824080 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:35.824588 (+ 508us) server_negotiation.cc:299] Negotiation successful
0807 18:33:35.824712 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260807 18:33:35.825642 18448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:35.812961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:45409 (local address 127.11.14.67:41201)
0807 18:33:35.813571 (+ 610us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:35.813632 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:35.815291 (+ 1659us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:35.815916 (+ 625us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:35.815926 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:35.816305 (+ 379us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:33:35.816802 (+ 497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:35.816816 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:35.817569 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:35.817571 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:35.818474 (+ 903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:35.818483 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:35.818939 (+ 456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:35.818941 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:35.819081 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:35.819754 (+ 673us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:33:35.819771 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:33:35.821321 (+ 1550us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:33:35.822864 (+ 1543us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:33:35.822869 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:33:35.822884 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:33:35.823146 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:33:35.823447 (+ 301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:33:35.823450 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:33:35.823452 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:33:35.823674 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:33:35.824081 (+ 407us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:33:35.824088 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:33:35.824341 (+ 253us) client_negotiation.cc:769] Sending connection context
0807 18:33:35.824527 (+ 186us) client_negotiation.cc:240] Negotiation successful
0807 18:33:35.824845 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":345,"thread_start_us":147,"threads_started":1}
I20260807 18:33:35.826776 18446 heartbeater.cc:347] Connected to a master server at 127.11.14.126:45409
I20260807 18:33:35.827039 18446 heartbeater.cc:464] Registering TS with master...
I20260807 18:33:35.827687 18446 heartbeater.cc:511] Master 127.11.14.126:45409 requested a full tablet report, sending...
I20260807 18:33:35.828769 17963 ts_manager.cc:194] Registered new tserver with Master: 6a3774b03adf44d3b1475982c04da6aa (127.11.14.67:42159)
I20260807 18:33:35.829303 17963 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.11.14.67@KRBTEST.COM'} at 127.11.14.67:41201
I20260807 18:33:35.840322 11321 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.11.14.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.14.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.14.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/krb5kdc/oryx_127.11.14.125.keytab.
Entry for principal oryx/127.11.14.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/krb5kdc/oryx_127.11.14.125.keytab.
I20260807 18:33:35.870966 11321 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkz56XN/build/debug/bin/kudu
/tmp/dist-test-taskkz56XN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.14.125:39483
--webserver_interface=127.11.14.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/krb5kdc/oryx_127.11.14.125.keytab
--principal=oryx/127.11.14.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.14.84:41437
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.14.126:45409,127.11.14.125:39483
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:33:35.987391 18451 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:33:35.987722 18451 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:33:35.987810 18451 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:33:35.991806 18451 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:33:35.991880 18451 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:33:35.991919 18451 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:33:35.991969 18451 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:33:35.992009 18451 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:33:35.997133 18451 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.14.84:41437
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.11.14.126:45409,127.11.14.125:39483
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/krb5kdc/oryx_127.11.14.125.keytab
--principal=oryx/127.11.14.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.14.125:39483
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.11.14.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18451
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
W20260807 18:33:35.997377 18451 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.11.14.126:45409,127.11.14.125:39483'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260807 18:33:35.998235 18451 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:33:35.999258 18451 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:33:36.008129 18460 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:33:36.008224 18462 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:33:36.008142 18459 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:33:36.008339 18451 server_base.cc:1034] running on GCE node
I20260807 18:33:36.009153 18451 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:33:36.010417 18451 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:33:36.011698 18451 hybrid_clock.cc:648] HybridClock initialized: now 1786127616011684 us; error 30 us; skew 500 ppm
Aug 07 18:33:36 dist-test-slave-6k22 krb5kdc[17915](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127616, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.14.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:33:36.014624 18451 init.cc:377] Logged in from keytab as oryx/127.11.14.125@KRBTEST.COM (short username oryx)
I20260807 18:33:36.015551 18451 webserver.cc:460] Webserver started at http://127.11.14.125:40027/ using document root <none> and password file <none>
I20260807 18:33:36.016194 18451 fs_manager.cc:359] Metadata directory not provided
I20260807 18:33:36.016260 18451 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:33:36.016455 18451 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:33:36.018702 18451 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/minicluster-data/master-1/data/instance:
uuid: "1abe9cee5c844198bbc7d32fdfae4374"
format_stamp: "Formatted at 2026-08-07 18:33:36 on dist-test-slave-6k22"
server_key: "5364678defb6bc70ca171fe88afa5bbd"
server_key_iv: "bb7edc57c8b76d56b69f4deea87365c8"
server_key_version: "encryptionkey@0"
I20260807 18:33:36.019251 18451 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/minicluster-data/master-1/wal/instance:
uuid: "1abe9cee5c844198bbc7d32fdfae4374"
format_stamp: "Formatted at 2026-08-07 18:33:36 on dist-test-slave-6k22"
server_key: "5364678defb6bc70ca171fe88afa5bbd"
server_key_iv: "bb7edc57c8b76d56b69f4deea87365c8"
server_key_version: "encryptionkey@0"
I20260807 18:33:36.023326 18451 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260807 18:33:36.025823 18471 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:33:36.027241 18451 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 18:33:36.027362 18451 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/minicluster-data/master-1/data,/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/minicluster-data/master-1/wal
uuid: "1abe9cee5c844198bbc7d32fdfae4374"
format_stamp: "Formatted at 2026-08-07 18:33:36 on dist-test-slave-6k22"
server_key: "5364678defb6bc70ca171fe88afa5bbd"
server_key_iv: "bb7edc57c8b76d56b69f4deea87365c8"
server_key_version: "encryptionkey@0"
I20260807 18:33:36.027496 18451 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:33:36.059367 18451 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:33:36.062865 18451 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:33:36.063040 18451 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:33:36.070792 18451 rpc_server.cc:307] RPC server started. Bound to: 127.11.14.125:39483
I20260807 18:33:36.071094 18527 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.14.125:39483 every 8 connection(s)
I20260807 18:33:36.071975 18451 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/minicluster-data/master-1/data/info.pb
I20260807 18:33:36.074821 18528 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:33:36.078536 11321 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkz56XN/build/debug/bin/kudu as pid 18451
I20260807 18:33:36.078748 18528 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.14.126" port: 45409 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 18:33:36 dist-test-slave-6k22 krb5kdc[17915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127616, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.14.125@KRBTEST.COM for oryx/127.11.14.126@KRBTEST.COM
Aug 07 18:33:36 dist-test-slave-6k22 krb5kdc[17915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127614, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.11.14.125@KRBTEST.COM
I20260807 18:33:36.093283 18004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:36.081133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32950 (local address 127.11.14.126:45409)
0807 18:33:36.081269 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:36.081275 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:36.081547 (+ 272us) server_negotiation.cc:407] Connection header received
0807 18:33:36.082005 (+ 458us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:36.082012 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:36.082074 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:36.082160 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:33:36.083354 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:36.084180 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:36.085743 (+ 1563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:36.086079 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:36.086175 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:36.088912 (+ 2737us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:33:36.088916 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:33:36.088919 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:33:36.088949 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:33:36.090833 (+ 1884us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:36.091598 (+ 765us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:36.091604 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:36.091609 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:36.091696 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:36.092090 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:36.092093 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:36.092095 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:36.092359 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:33:36.092430 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:36.092992 (+ 562us) server_negotiation.cc:299] Negotiation successful
0807 18:33:36.093128 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260807 18:33:36.094362 18532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:36.081045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:45409 (local address 127.0.0.1:32950)
0807 18:33:36.081390 (+ 345us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:36.081443 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:36.081770 (+ 327us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:36.082252 (+ 482us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:36.082259 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:36.082786 (+ 527us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:33:36.083190 (+ 404us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:36.083207 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:36.084369 (+ 1162us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:36.084373 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:36.085606 (+ 1233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:36.085617 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:36.086296 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:36.086300 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:36.086490 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:36.086989 (+ 499us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:33:36.087006 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:33:36.088703 (+ 1697us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:33:36.090952 (+ 2249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:33:36.090961 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:33:36.090971 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:33:36.091440 (+ 469us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:33:36.091805 (+ 365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:33:36.091808 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:33:36.091810 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:33:36.091976 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:33:36.092516 (+ 540us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:33:36.092522 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:33:36.092740 (+ 218us) client_negotiation.cc:769] Sending connection context
0807 18:33:36.093682 (+ 942us) client_negotiation.cc:240] Negotiation successful
0807 18:33:36.093879 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":177,"spinlock_wait_cycles":896,"thread_start_us":109,"threads_started":1}
I20260807 18:33:36.094363 18531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:36.080360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54210 (local address 127.11.14.125:39483)
0807 18:33:36.080758 (+ 398us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:36.080772 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:36.080807 (+ 35us) server_negotiation.cc:407] Connection header received
0807 18:33:36.081663 (+ 856us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:36.081683 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:36.081897 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:36.082167 (+ 270us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:33:36.083179 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:36.084551 (+ 1372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:36.085420 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:36.085900 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:36.086055 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:36.089438 (+ 3383us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:33:36.089448 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:33:36.089462 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:33:36.089486 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:33:36.091409 (+ 1923us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:36.091955 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:36.091961 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:36.091963 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:36.092051 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:36.092347 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:36.092350 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:36.092352 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:36.092713 (+ 361us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:33:36.092954 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:36.093189 (+ 235us) server_negotiation.cc:299] Negotiation successful
0807 18:33:36.093494 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":265,"thread_start_us":113,"threads_started":1}
I20260807 18:33:36.096258 18528 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.14.125" port: 39483 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 18:33:36 dist-test-slave-6k22 krb5kdc[17915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127616, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.14.125@KRBTEST.COM for oryx/127.11.14.125@KRBTEST.COM
I20260807 18:33:36.105191 18531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:36.097151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54218 (local address 127.11.14.125:39483)
0807 18:33:36.097270 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:36.097274 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:36.097286 (+ 12us) server_negotiation.cc:407] Connection header received
0807 18:33:36.097318 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:36.097321 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:36.097372 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:36.097453 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:33:36.097801 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:36.099032 (+ 1231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:36.099897 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:36.100138 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:36.100253 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:36.102231 (+ 1978us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:33:36.102234 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:33:36.102236 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:33:36.102256 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:33:36.103480 (+ 1224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:36.103999 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:36.104004 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:36.104006 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:36.104080 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:36.104356 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:36.104361 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:36.104362 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:36.104525 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:33:36.104638 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:36.104868 (+ 230us) server_negotiation.cc:299] Negotiation successful
0807 18:33:36.105016 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":58,"server-negotiator.queue_time_us":46}
I20260807 18:33:36.105191 18532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:36.096795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.125:39483 (local address 127.0.0.1:54218)
0807 18:33:36.096910 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:36.096927 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:36.096997 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:36.097480 (+ 483us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:36.097482 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:36.097610 (+ 128us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:33:36.097699 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:36.097704 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:36.099182 (+ 1478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:36.099185 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:36.099756 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:36.099765 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:36.100249 (+ 484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:36.100252 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:36.100344 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:36.100894 (+ 550us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:33:36.100906 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:33:36.102089 (+ 1183us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:33:36.103705 (+ 1616us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:33:36.103708 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:33:36.103710 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:33:36.103898 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:33:36.104172 (+ 274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:33:36.104176 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:33:36.104178 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:33:36.104266 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:33:36.104620 (+ 354us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:33:36.104623 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:33:36.104708 (+ 85us) client_negotiation.cc:769] Sending connection context
0807 18:33:36.104823 (+ 115us) client_negotiation.cc:240] Negotiation successful
0807 18:33:36.104962 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48}
I20260807 18:33:36.108234 18528 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1abe9cee5c844198bbc7d32fdfae4374: Bootstrap starting.
I20260807 18:33:36.110787 18528 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1abe9cee5c844198bbc7d32fdfae4374: Neither blocks nor log segments found. Creating new log.
I20260807 18:33:36.111668 18528 log.cc:824] T 00000000000000000000000000000000 P 1abe9cee5c844198bbc7d32fdfae4374: Log is configured to *not* fsync() on all Append() calls
I20260807 18:33:36.113682 18528 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1abe9cee5c844198bbc7d32fdfae4374: No bootstrap required, opened a new log
I20260807 18:33:36.116788 18528 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1abe9cee5c844198bbc7d32fdfae4374 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b176ea138c9402a95ef84541f6cc5e3" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 45409 } } peers { permanent_uuid: "1abe9cee5c844198bbc7d32fdfae4374" member_type: VOTER last_known_addr { host: "127.11.14.125" port: 39483 } }
I20260807 18:33:36.117005 18528 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1abe9cee5c844198bbc7d32fdfae4374 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:33:36.117041 18528 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1abe9cee5c844198bbc7d32fdfae4374 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1abe9cee5c844198bbc7d32fdfae4374, State: Initialized, Role: FOLLOWER
I20260807 18:33:36.117508 18528 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1abe9cee5c844198bbc7d32fdfae4374 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b176ea138c9402a95ef84541f6cc5e3" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 45409 } } peers { permanent_uuid: "1abe9cee5c844198bbc7d32fdfae4374" member_type: VOTER last_known_addr { host: "127.11.14.125" port: 39483 } }
I20260807 18:33:36.117995 18534 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1abe9cee5c844198bbc7d32fdfae4374 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b176ea138c9402a95ef84541f6cc5e3" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 45409 } } peers { permanent_uuid: "1abe9cee5c844198bbc7d32fdfae4374" member_type: VOTER last_known_addr { host: "127.11.14.125" port: 39483 } } }
I20260807 18:33:36.118120 18534 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1abe9cee5c844198bbc7d32fdfae4374 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 18:33:36.118821 18528 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1abe9cee5c844198bbc7d32fdfae4374 [sys.catalog]: configured and running, proceeding with master startup.
W20260807 18:33:36.123724 18547 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1abe9cee5c844198bbc7d32fdfae4374: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 18:33:36.123808 18547 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 18:33:36.125813 18451 master_runner.cc:428] Detected that this master 1abe9cee5c844198bbc7d32fdfae4374 is joining an existing cluster
I20260807 18:33:36.125873 18451 master_runner.cc:432] Initiating AddMaster RPC to add 127.11.14.125:39483
I20260807 18:33:36.137861 18531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:36.128500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54224 (local address 127.11.14.125:39483)
0807 18:33:36.128609 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:36.128614 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:36.128627 (+ 13us) server_negotiation.cc:407] Connection header received
0807 18:33:36.128662 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:36.128665 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:36.128708 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:36.128792 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:33:36.129473 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:36.130224 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:36.131492 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:36.131879 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:36.132022 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:36.134161 (+ 2139us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:33:36.134165 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:33:36.134167 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:33:36.134200 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:33:36.136037 (+ 1837us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:36.136563 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:36.136566 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:36.136568 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:36.136658 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:36.136947 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:36.136950 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:36.136952 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:36.137152 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:33:36.137255 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:36.137518 (+ 263us) server_negotiation.cc:299] Negotiation successful
0807 18:33:36.137679 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":66,"server-negotiator.queue_time_us":43,"spinlock_wait_cycles":3968}
I20260807 18:33:36.137869 18004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:36.128384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32958 (local address 127.11.14.126:45409)
0807 18:33:36.128643 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:36.128648 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:36.128659 (+ 11us) server_negotiation.cc:407] Connection header received
0807 18:33:36.128694 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:36.128697 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:36.128741 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:36.128813 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:33:36.129407 (+ 594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:36.130236 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:36.132639 (+ 2403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:36.132892 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:36.132977 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:36.134380 (+ 1403us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:33:36.134386 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:33:36.134391 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:33:36.134426 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:33:36.136158 (+ 1732us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:36.136651 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:36.136654 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:36.136656 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:36.136742 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:36.137032 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:36.137036 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:36.137038 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:36.137261 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:33:36.137359 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:36.137617 (+ 258us) server_negotiation.cc:299] Negotiation successful
0807 18:33:36.137735 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":179}
I20260807 18:33:36.137864 18554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:36.127990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.125:39483 (local address 127.0.0.1:54224)
0807 18:33:36.128381 (+ 391us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:36.128395 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:36.128493 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:36.128838 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:36.128841 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:36.129149 (+ 308us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:33:36.129319 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:36.129328 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:36.130348 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:36.130351 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:36.131372 (+ 1021us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:36.131383 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:36.132031 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:36.132034 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:36.132119 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:36.133577 (+ 1458us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:33:36.133598 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:33:36.134028 (+ 430us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:33:36.136158 (+ 2130us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:33:36.136162 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:33:36.136164 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:33:36.136415 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:33:36.136756 (+ 341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:33:36.136759 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:33:36.136761 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:33:36.136851 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:33:36.137255 (+ 404us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:33:36.137258 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:33:36.137345 (+ 87us) client_negotiation.cc:769] Sending connection context
0807 18:33:36.137453 (+ 108us) client_negotiation.cc:240] Negotiation successful
0807 18:33:36.137615 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":324,"mutex_wait_us":21,"thread_start_us":123,"threads_started":1}
I20260807 18:33:36.138265 18553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:36.127941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:45409 (local address 127.0.0.1:32958)
0807 18:33:36.128244 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:36.128259 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:36.128355 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:36.128920 (+ 565us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:36.128923 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:36.129154 (+ 231us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:33:36.129276 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:36.129282 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:36.131522 (+ 2240us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:36.131525 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:36.132478 (+ 953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:36.132489 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:36.133003 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:36.133007 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:36.133130 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:36.133682 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:33:36.133699 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:33:36.134227 (+ 528us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:33:36.136263 (+ 2036us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:33:36.136266 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:33:36.136269 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:33:36.136537 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:33:36.136825 (+ 288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:33:36.136828 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:33:36.136830 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:33:36.136946 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:33:36.137348 (+ 402us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:33:36.137351 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:33:36.137421 (+ 70us) client_negotiation.cc:769] Sending connection context
0807 18:33:36.138020 (+ 599us) client_negotiation.cc:240] Negotiation successful
0807 18:33:36.138142 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":209,"mutex_wait_us":3,"spinlock_wait_cycles":4096,"thread_start_us":100,"threads_started":1}
Aug 07 18:33:36 dist-test-slave-6k22 krb5kdc[17915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127615, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.14.126@KRBTEST.COM for oryx/127.11.14.125@KRBTEST.COM
I20260807 18:33:36.153944 18531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:36.143972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54236 (local address 127.11.14.125:39483)
0807 18:33:36.144100 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:36.144105 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:36.144497 (+ 392us) server_negotiation.cc:407] Connection header received
0807 18:33:36.144666 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:36.144668 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:36.144710 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:36.144775 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:33:36.145530 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:36.145981 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:36.147211 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:36.147411 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:36.147526 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:36.150445 (+ 2919us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:33:36.150450 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:33:36.150456 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:33:36.150484 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:33:36.152011 (+ 1527us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:36.152488 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:36.152493 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:36.152495 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:36.152576 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:36.152916 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:36.152920 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:36.152921 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:36.153131 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:33:36.153225 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:36.153654 (+ 429us) server_negotiation.cc:299] Negotiation successful
0807 18:33:36.153766 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260807 18:33:36.153965 18555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:36.143951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.125:39483 (local address 127.0.0.1:54236)
0807 18:33:36.144330 (+ 379us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:36.144391 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:36.144540 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:36.144834 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:36.144840 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:36.145293 (+ 453us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:33:36.145420 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:36.145427 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:36.146082 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:36.146085 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:36.147094 (+ 1009us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:36.147105 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:36.147530 (+ 425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:36.147534 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:36.147669 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:36.148219 (+ 550us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:33:36.148234 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:33:36.150214 (+ 1980us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:33:36.152133 (+ 1919us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:33:36.152142 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:33:36.152149 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:33:36.152386 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:33:36.152668 (+ 282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:33:36.152672 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:33:36.152674 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:33:36.152763 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:33:36.153262 (+ 499us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:33:36.153269 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:33:36.153388 (+ 119us) client_negotiation.cc:769] Sending connection context
0807 18:33:36.153586 (+ 198us) client_negotiation.cc:240] Negotiation successful
0807 18:33:36.153780 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":106,"threads_started":1}
I20260807 18:33:36.155642 17963 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.11.14.125:39483: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "1abe9cee5c844198bbc7d32fdfae4374"
member_type: NON_VOTER
last_known_addr {
host: "127.11.14.125"
port: 39483
}
attrs {
promote: true
}
}
dest_uuid: "9b176ea138c9402a95ef84541f6cc5e3"
cas_config_opid_index: -1
I20260807 18:33:36.156363 17963 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9b176ea138c9402a95ef84541f6cc5e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "9b176ea138c9402a95ef84541f6cc5e3" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 45409 } } peers { permanent_uuid: "1abe9cee5c844198bbc7d32fdfae4374" member_type: NON_VOTER last_known_addr { host: "127.11.14.125" port: 39483 } attrs { promote: true } }
WARNING: no policy specified for oryx/127.11.14.124@KRBTEST.COM; defaulting to no policy
I20260807 18:33:36.158607 18559 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 9b176ea138c9402a95ef84541f6cc5e3 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 1abe9cee5c844198bbc7d32fdfae4374 (127.11.14.125) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "9b176ea138c9402a95ef84541f6cc5e3" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 45409 } } peers { permanent_uuid: "1abe9cee5c844198bbc7d32fdfae4374" member_type: NON_VOTER last_known_addr { host: "127.11.14.125" port: 39483 } attrs { promote: true } } }
I20260807 18:33:36.159224 18499 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1abe9cee5c844198bbc7d32fdfae4374 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:33:36.161651 18559 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.11.14.125:39483
I20260807 18:33:36.161728 18499 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1abe9cee5c844198bbc7d32fdfae4374 [term 1 FOLLOWER]: Refusing update from remote peer 9b176ea138c9402a95ef84541f6cc5e3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.14.124@KRBTEST.COM" created.
I20260807 18:33:36.161921 18559 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b176ea138c9402a95ef84541f6cc5e3 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "9b176ea138c9402a95ef84541f6cc5e3" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "9b176ea138c9402a95ef84541f6cc5e3" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 45409 } } peers { permanent_uuid: "1abe9cee5c844198bbc7d32fdfae4374" member_type: NON_VOTER last_known_addr { host: "127.11.14.125" port: 39483 } attrs { promote: true } } }
I20260807 18:33:36.162009 18559 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b176ea138c9402a95ef84541f6cc5e3 [sys.catalog]: This master's current role is: LEADER
I20260807 18:33:36.162603 18451 master.cc:537] Master@127.11.14.125:39483 shutting down...
I20260807 18:33:36.162650 18559 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 9b176ea138c9402a95ef84541f6cc5e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1abe9cee5c844198bbc7d32fdfae4374" member_type: NON_VOTER last_known_addr { host: "127.11.14.125" port: 39483 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260807 18:33:36.165078 17946 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260807 18:33:36.167039 17946 consensus_peers.cc:487] T 00000000000000000000000000000000 P 9b176ea138c9402a95ef84541f6cc5e3 -> Peer 1abe9cee5c844198bbc7d32fdfae4374 (127.11.14.125:39483): Couldn't send request to peer 1abe9cee5c844198bbc7d32fdfae4374. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260807 18:33:36.168069 18451 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 1abe9cee5c844198bbc7d32fdfae4374 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:33:36.168378 18451 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 1abe9cee5c844198bbc7d32fdfae4374 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:33:36.168522 18451 tablet_replica.cc:330] T 00000000000000000000000000000000 P 1abe9cee5c844198bbc7d32fdfae4374: stopping tablet replica
I20260807 18:33:36.173966 18451 master.cc:552] Master@127.11.14.125:39483 shutdown complete.
I20260807 18:33:36.174018 18451 master_runner.cc:305] Clearing existing system tablet
I20260807 18:33:36.174295 18451 fs_manager.cc:359] Metadata directory not provided
I20260807 18:33:36.174386 18451 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:33:36.176091 18451 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 18:33:36.177160 18564 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:33:36.177402 18451 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:33:36.177541 18451 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 18:33:36.177603 18451 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/minicluster-data/master-1/data,/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/minicluster-data/master-1/wal
uuid: "1abe9cee5c844198bbc7d32fdfae4374"
format_stamp: "Formatted at 2026-08-07 18:33:36 on dist-test-slave-6k22"
server_key: "5364678defb6bc70ca171fe88afa5bbd"
server_key_iv: "bb7edc57c8b76d56b69f4deea87365c8"
server_key_version: "encryptionkey@0"
I20260807 18:33:36.179979 18451 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 1abe9cee5c844198bbc7d32fdfae4374: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:33:36.182724 18451 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 1abe9cee5c844198bbc7d32fdfae4374: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260807 18:33:36.182883 18451 log.cc:1196] T 00000000000000000000000000000000 P 1abe9cee5c844198bbc7d32fdfae4374: Deleting WAL directory at /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260807 18:33:36.183246 18451 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 1abe9cee5c844198bbc7d32fdfae4374: Deleting consensus metadata
I20260807 18:33:36.183498 18451 master_runner.cc:315] Copying system tablet from 127.11.14.126:45409
I20260807 18:33:36.184804 18451 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 1abe9cee5c844198bbc7d32fdfae4374: tablet copy: Beginning tablet copy session from remote peer at address 127.11.14.126:45409
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.14.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/krb5kdc/oryx_127.11.14.124.keytab.
Entry for principal oryx/127.11.14.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/krb5kdc/oryx_127.11.14.124.keytab.
I20260807 18:33:36.191633 11321 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkz56XN/build/debug/bin/kudu
/tmp/dist-test-taskkz56XN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.14.124:36503
--webserver_interface=127.11.14.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/krb5kdc/oryx_127.11.14.124.keytab
--principal=oryx/127.11.14.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.14.84:41437
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.14.126:45409,127.11.14.125:39483,127.11.14.124:36503
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 18:33:36.193794 18004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:36.185500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32968 (local address 127.11.14.126:45409)
0807 18:33:36.185651 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:36.185656 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:36.185815 (+ 159us) server_negotiation.cc:407] Connection header received
0807 18:33:36.185987 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:36.185990 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:36.186042 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:36.186123 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:33:36.186593 (+ 470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:36.187189 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:36.188159 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:36.188401 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:36.188517 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:36.190028 (+ 1511us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:33:36.190032 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:33:36.190035 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:33:36.190060 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:33:36.191873 (+ 1813us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:36.192391 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:36.192398 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:36.192403 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:36.192501 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:36.192807 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:36.192813 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:36.192818 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:36.193081 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:33:36.193217 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:36.193509 (+ 292us) server_negotiation.cc:299] Negotiation successful
0807 18:33:36.193624 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260807 18:33:36.194310 18570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:36.185380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:45409 (local address 127.0.0.1:32968)
0807 18:33:36.185712 (+ 332us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:36.185726 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:36.185877 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:36.186153 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:36.186156 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:36.186370 (+ 214us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:33:36.186466 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:36.186472 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:36.187306 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:36.187311 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:36.188024 (+ 713us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:36.188035 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:36.188636 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:36.188638 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:36.188698 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:36.189455 (+ 757us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:33:36.189468 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:33:36.189880 (+ 412us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:33:36.192002 (+ 2122us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:33:36.192005 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:33:36.192007 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:33:36.192274 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:33:36.192609 (+ 335us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:33:36.192612 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:33:36.192614 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:33:36.192708 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:33:36.193204 (+ 496us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:33:36.193207 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:33:36.193306 (+ 99us) client_negotiation.cc:769] Sending connection context
0807 18:33:36.193981 (+ 675us) client_negotiation.cc:240] Negotiation successful
0807 18:33:36.194131 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":105,"threads_started":1}
I20260807 18:33:36.194931 17984 tablet_copy_service.cc:140] P 9b176ea138c9402a95ef84541f6cc5e3: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 1abe9cee5c844198bbc7d32fdfae4374 ({username='oryx', principal='oryx/127.11.14.125@KRBTEST.COM'} at 127.0.0.1:32968)
I20260807 18:33:36.195145 17984 tablet_copy_service.cc:161] P 9b176ea138c9402a95ef84541f6cc5e3: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 1abe9cee5c844198bbc7d32fdfae4374 at {username='oryx', principal='oryx/127.11.14.125@KRBTEST.COM'} at 127.0.0.1:32968: session id = 1abe9cee5c844198bbc7d32fdfae4374-00000000000000000000000000000000
I20260807 18:33:36.197063 17984 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 9b176ea138c9402a95ef84541f6cc5e3: Tablet Copy: opened 0 blocks and 1 log segments
I20260807 18:33:36.198922 18451 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260807 18:33:36.201206 18451 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 1abe9cee5c844198bbc7d32fdfae4374: tablet copy: Starting download of 0 data blocks...
I20260807 18:33:36.201390 18451 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 1abe9cee5c844198bbc7d32fdfae4374: tablet copy: Starting download of 1 WAL segments...
I20260807 18:33:36.203291 18451 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 1abe9cee5c844198bbc7d32fdfae4374: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260807 18:33:36.205086 17984 tablet_copy_service.cc:342] P 9b176ea138c9402a95ef84541f6cc5e3: Request end of tablet copy session 1abe9cee5c844198bbc7d32fdfae4374-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.11.14.125@KRBTEST.COM'} at 127.0.0.1:32968
I20260807 18:33:36.205207 17984 tablet_copy_service.cc:433] P 9b176ea138c9402a95ef84541f6cc5e3: ending tablet copy session 1abe9cee5c844198bbc7d32fdfae4374-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 1abe9cee5c844198bbc7d32fdfae4374
I20260807 18:33:36.212993 18451 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:33:36.214745 18572 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:33:36.216960 18451 server_base.cc:1034] running on GCE node
W20260807 18:33:36.216888 18576 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:33:36.220095 18573 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:33:36.220320 18451 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:33:36.220629 18451 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:33:36.221801 18451 hybrid_clock.cc:648] HybridClock initialized: now 1786127616221788 us; error 30 us; skew 500 ppm
Aug 07 18:33:36 dist-test-slave-6k22 krb5kdc[17915](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127616, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.14.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:33:36.223438 18451 init.cc:377] Logged in from keytab as oryx/127.11.14.125@KRBTEST.COM (short username oryx)
I20260807 18:33:36.223928 18451 webserver.cc:460] Webserver started at http://127.11.14.125:38573/ using document root <none> and password file <none>
I20260807 18:33:36.224116 18451 fs_manager.cc:359] Metadata directory not provided
I20260807 18:33:36.224171 18451 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:33:36.225572 18451 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 18:33:36.226694 18583 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:33:36.227100 18451 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 18:33:36.227219 18451 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/minicluster-data/master-1/data,/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/minicluster-data/master-1/wal
uuid: "1abe9cee5c844198bbc7d32fdfae4374"
format_stamp: "Formatted at 2026-08-07 18:33:36 on dist-test-slave-6k22"
server_key: "5364678defb6bc70ca171fe88afa5bbd"
server_key_iv: "bb7edc57c8b76d56b69f4deea87365c8"
server_key_version: "encryptionkey@0"
I20260807 18:33:36.227362 18451 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:33:36.233106 18451 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:33:36.235214 18451 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:33:36.240087 18451 rpc_server.cc:307] RPC server started. Bound to: 127.11.14.125:39483
I20260807 18:33:36.240358 18640 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.14.125:39483 every 8 connection(s)
I20260807 18:33:36.240764 18451 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/minicluster-data/master-1/data/info.pb
I20260807 18:33:36.242554 18641 sys_catalog.cc:263] Verifying existing consensus state
I20260807 18:33:36.243696 18641 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1abe9cee5c844198bbc7d32fdfae4374: Bootstrap starting.
I20260807 18:33:36.255121 18641 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1abe9cee5c844198bbc7d32fdfae4374: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 18:33:36.255445 18641 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1abe9cee5c844198bbc7d32fdfae4374: Bootstrap complete.
I20260807 18:33:36.256042 18641 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1abe9cee5c844198bbc7d32fdfae4374 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "9b176ea138c9402a95ef84541f6cc5e3" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 45409 } } peers { permanent_uuid: "1abe9cee5c844198bbc7d32fdfae4374" member_type: NON_VOTER last_known_addr { host: "127.11.14.125" port: 39483 } attrs { promote: true } }
I20260807 18:33:36.256182 18641 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1abe9cee5c844198bbc7d32fdfae4374 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 1abe9cee5c844198bbc7d32fdfae4374, State: Initialized, Role: LEARNER
I20260807 18:33:36.256285 18641 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1abe9cee5c844198bbc7d32fdfae4374 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "9b176ea138c9402a95ef84541f6cc5e3" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 45409 } } peers { permanent_uuid: "1abe9cee5c844198bbc7d32fdfae4374" member_type: NON_VOTER last_known_addr { host: "127.11.14.125" port: 39483 } attrs { promote: true } }
I20260807 18:33:36.256620 18644 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1abe9cee5c844198bbc7d32fdfae4374 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "9b176ea138c9402a95ef84541f6cc5e3" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 45409 } } peers { permanent_uuid: "1abe9cee5c844198bbc7d32fdfae4374" member_type: NON_VOTER last_known_addr { host: "127.11.14.125" port: 39483 } attrs { promote: true } } }
I20260807 18:33:36.256726 18641 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1abe9cee5c844198bbc7d32fdfae4374 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:33:36.256809 18644 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1abe9cee5c844198bbc7d32fdfae4374 [sys.catalog]: This master's current role is: LEARNER
I20260807 18:33:36.260718 18656 catalog_manager.cc:1259] Loaded cluster ID: b84b754b281a4b06b86e0ce85d0fbbe8
I20260807 18:33:36.260780 18656 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 1abe9cee5c844198bbc7d32fdfae4374: loading cluster ID for follower catalog manager: success
I20260807 18:33:36.262881 18656 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 1abe9cee5c844198bbc7d32fdfae4374: acquiring CA information for follower catalog manager: success
I20260807 18:33:36.264312 18656 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 1abe9cee5c844198bbc7d32fdfae4374: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260807 18:33:36.328827 18571 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:33:36.329119 18571 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:33:36.329200 18571 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:33:36.333261 18571 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:33:36.333348 18571 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:33:36.333386 18571 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:33:36.333443 18571 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:33:36.333482 18571 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:33:36.339071 18571 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.14.84:41437
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.11.14.126:45409,127.11.14.125:39483,127.11.14.124:36503
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/krb5kdc/oryx_127.11.14.124.keytab
--principal=oryx/127.11.14.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.14.124:36503
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.11.14.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18571
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:33:36.340173 18571 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:33:36.341215 18571 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:33:36.349957 18664 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:33:36.349957 18662 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:33:36.350039 18571 server_base.cc:1034] running on GCE node
W20260807 18:33:36.349988 18661 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:33:36.350752 18571 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:33:36.351984 18571 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:33:36.353181 18571 hybrid_clock.cc:648] HybridClock initialized: now 1786127616353149 us; error 41 us; skew 500 ppm
Aug 07 18:33:36 dist-test-slave-6k22 krb5kdc[17915](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127616, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.14.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:33:36.356382 18571 init.cc:377] Logged in from keytab as oryx/127.11.14.124@KRBTEST.COM (short username oryx)
I20260807 18:33:36.357676 18571 webserver.cc:460] Webserver started at http://127.11.14.124:38497/ using document root <none> and password file <none>
I20260807 18:33:36.358539 18571 fs_manager.cc:359] Metadata directory not provided
I20260807 18:33:36.358700 18571 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:33:36.358973 18571 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:33:36.361469 18571 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/minicluster-data/master-2/data/instance:
uuid: "ab43b0280a964c18a81ab0531a0a2a10"
format_stamp: "Formatted at 2026-08-07 18:33:36 on dist-test-slave-6k22"
server_key: "2d34986feb2bf6cd49bcf9ca715c6a2e"
server_key_iv: "492e11eafdd5e2f41aeea36a0e406435"
server_key_version: "encryptionkey@0"
I20260807 18:33:36.362104 18571 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/minicluster-data/master-2/wal/instance:
uuid: "ab43b0280a964c18a81ab0531a0a2a10"
format_stamp: "Formatted at 2026-08-07 18:33:36 on dist-test-slave-6k22"
server_key: "2d34986feb2bf6cd49bcf9ca715c6a2e"
server_key_iv: "492e11eafdd5e2f41aeea36a0e406435"
server_key_version: "encryptionkey@0"
I20260807 18:33:36.365947 18571 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 18:33:36.368399 18672 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:33:36.369750 18571 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260807 18:33:36.369884 18571 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/minicluster-data/master-2/data,/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/minicluster-data/master-2/wal
uuid: "ab43b0280a964c18a81ab0531a0a2a10"
format_stamp: "Formatted at 2026-08-07 18:33:36 on dist-test-slave-6k22"
server_key: "2d34986feb2bf6cd49bcf9ca715c6a2e"
server_key_iv: "492e11eafdd5e2f41aeea36a0e406435"
server_key_version: "encryptionkey@0"
I20260807 18:33:36.369994 18571 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:33:36.381139 18163 heartbeater.cc:503] Master 127.11.14.126:45409 was elected leader, sending a full tablet report...
I20260807 18:33:36.390087 18571 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:33:36.393560 18571 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:33:36.393769 18571 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:33:36.400986 18571 rpc_server.cc:307] RPC server started. Bound to: 127.11.14.124:36503
I20260807 18:33:36.401279 18725 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.14.124:36503 every 8 connection(s)
I20260807 18:33:36.402033 18571 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/minicluster-data/master-2/data/info.pb
I20260807 18:33:36.405002 18726 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:33:36.409124 18726 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.14.126" port: 45409 } has no permanent_uuid. Determining permanent_uuid...
I20260807 18:33:36.409838 11321 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkz56XN/build/debug/bin/kudu as pid 18571
Aug 07 18:33:36 dist-test-slave-6k22 krb5kdc[17915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127616, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.14.124@KRBTEST.COM for oryx/127.11.14.126@KRBTEST.COM
Aug 07 18:33:36 dist-test-slave-6k22 krb5kdc[17915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127614, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.11.14.124@KRBTEST.COM
I20260807 18:33:36.424278 18004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:36.411312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32976 (local address 127.11.14.126:45409)
0807 18:33:36.411462 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:36.411470 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:36.412032 (+ 562us) server_negotiation.cc:407] Connection header received
0807 18:33:36.412751 (+ 719us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:36.412755 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:36.412802 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:36.412901 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:33:36.414025 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:36.414837 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:36.416293 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:36.416627 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:36.416738 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:36.419799 (+ 3061us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:33:36.419803 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:33:36.419805 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:33:36.419831 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:33:36.421732 (+ 1901us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:36.422458 (+ 726us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:36.422461 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:36.422462 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:36.422575 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:36.422939 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:36.422942 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:36.422943 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:36.423228 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:33:36.423326 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:36.423937 (+ 611us) server_negotiation.cc:299] Negotiation successful
0807 18:33:36.424067 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260807 18:33:36.425170 18728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:36.411326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44540 (local address 127.11.14.124:36503)
0807 18:33:36.411773 (+ 447us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:36.411788 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:36.411823 (+ 35us) server_negotiation.cc:407] Connection header received
0807 18:33:36.412527 (+ 704us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:36.412545 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:36.412791 (+ 246us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:36.412962 (+ 171us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:33:36.413862 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:36.415139 (+ 1277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:36.416293 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:36.416876 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:36.416955 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:36.420281 (+ 3326us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:33:36.420304 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:33:36.420320 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:33:36.420346 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:33:36.422449 (+ 2103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:36.422911 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:36.422918 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:36.422920 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:36.423010 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:36.423324 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:36.423328 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:36.423329 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:36.423671 (+ 342us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:33:36.423880 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:36.424199 (+ 319us) server_negotiation.cc:299] Negotiation successful
0807 18:33:36.424321 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":92,"server-negotiator.queue_time_us":301,"thread_start_us":149,"threads_started":1}
I20260807 18:33:36.425170 18729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:36.411325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:45409 (local address 127.0.0.1:32976)
0807 18:33:36.411873 (+ 548us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:36.411923 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:36.412542 (+ 619us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:36.412985 (+ 443us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:36.412991 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:36.413492 (+ 501us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:33:36.413883 (+ 391us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:36.413898 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:36.414992 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:36.414994 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:36.416178 (+ 1184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:36.416189 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:36.416778 (+ 589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:36.416781 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:36.416950 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:36.417694 (+ 744us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:33:36.417716 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:33:36.419626 (+ 1910us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:33:36.421854 (+ 2228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:33:36.421863 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:33:36.421871 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:33:36.422341 (+ 470us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:33:36.422676 (+ 335us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:33:36.422678 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:33:36.422679 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:33:36.422813 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:33:36.423342 (+ 529us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:33:36.423347 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:33:36.423668 (+ 321us) client_negotiation.cc:769] Sending connection context
0807 18:33:36.423864 (+ 196us) client_negotiation.cc:240] Negotiation successful
0807 18:33:36.424196 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":312,"mutex_wait_us":25,"spinlock_wait_cycles":23936,"thread_start_us":181,"threads_started":1}
I20260807 18:33:36.426440 18726 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.14.125" port: 39483 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 18:33:36 dist-test-slave-6k22 krb5kdc[17915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127616, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.14.124@KRBTEST.COM for oryx/127.11.14.125@KRBTEST.COM
I20260807 18:33:36.435087 18729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:36.426942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.125:39483 (local address 127.0.0.1:54252)
0807 18:33:36.427089 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:36.427099 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:36.427155 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:36.427635 (+ 480us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:36.427638 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:36.427824 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:33:36.427916 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:36.427921 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:36.428943 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:36.428946 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:36.429455 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:36.429462 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:36.430136 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:36.430138 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:36.430213 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:36.430586 (+ 373us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:33:36.430596 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:33:36.431756 (+ 1160us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:33:36.433714 (+ 1958us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:33:36.433716 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:33:36.433718 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:33:36.433888 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:33:36.434186 (+ 298us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:33:36.434188 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:33:36.434189 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:33:36.434259 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:33:36.434642 (+ 383us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:33:36.434644 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:33:36.434716 (+ 72us) client_negotiation.cc:769] Sending connection context
0807 18:33:36.434796 (+ 80us) client_negotiation.cc:240] Negotiation successful
0807 18:33:36.434948 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":94}
I20260807 18:33:36.435103 18730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:36.427042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54252 (local address 127.11.14.125:39483)
0807 18:33:36.427341 (+ 299us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:36.427345 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:36.427358 (+ 13us) server_negotiation.cc:407] Connection header received
0807 18:33:36.427395 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:36.427396 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:36.427465 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:36.427593 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:33:36.428054 (+ 461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:36.428826 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:36.429560 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:36.429845 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:36.430148 (+ 303us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:36.431913 (+ 1765us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:33:36.431916 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:33:36.431918 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:33:36.431944 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:33:36.433606 (+ 1662us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:36.433987 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:36.433990 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:36.433991 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:36.434081 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:36.434369 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:36.434372 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:36.434373 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:36.434554 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:33:36.434650 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:36.434884 (+ 234us) server_negotiation.cc:299] Negotiation successful
0807 18:33:36.434984 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":227,"thread_start_us":110,"threads_started":1}
I20260807 18:33:36.435921 18726 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.14.124" port: 36503 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 18:33:36 dist-test-slave-6k22 krb5kdc[17915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127616, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.14.124@KRBTEST.COM for oryx/127.11.14.124@KRBTEST.COM
I20260807 18:33:36.443150 18729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:36.436368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.124:36503 (local address 127.0.0.1:44542)
0807 18:33:36.436519 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:36.436533 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:36.436619 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:36.436849 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:36.436852 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:36.437008 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:33:36.437066 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:36.437071 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:36.437789 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:36.437795 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:36.438279 (+ 484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:36.438286 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:36.438771 (+ 485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:36.438774 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:36.438850 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:36.439257 (+ 407us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:33:36.439268 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:33:36.440474 (+ 1206us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:33:36.441883 (+ 1409us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:33:36.441886 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:33:36.441887 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:33:36.442066 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:33:36.442304 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:33:36.442306 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:33:36.442307 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:33:36.442380 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:33:36.442719 (+ 339us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:33:36.442721 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:33:36.442793 (+ 72us) client_negotiation.cc:769] Sending connection context
0807 18:33:36.442874 (+ 81us) client_negotiation.cc:240] Negotiation successful
0807 18:33:36.443020 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51}
I20260807 18:33:36.443207 18728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:36.436458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44542 (local address 127.11.14.124:36503)
0807 18:33:36.436543 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:36.436550 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:36.436581 (+ 31us) server_negotiation.cc:407] Connection header received
0807 18:33:36.436714 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:36.436717 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:36.436751 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:36.436848 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:33:36.437174 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:36.437681 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:36.438415 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:36.438662 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:36.438772 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:36.440601 (+ 1829us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:33:36.440604 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:33:36.440606 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:33:36.440627 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:33:36.441783 (+ 1156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:36.442148 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:36.442150 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:36.442151 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:36.442220 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:36.442458 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:36.442460 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:36.442461 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:36.442609 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:33:36.442683 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:36.442949 (+ 266us) server_negotiation.cc:299] Negotiation successful
0807 18:33:36.443070 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":30,"server-negotiator.queue_time_us":33}
I20260807 18:33:36.446902 18726 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ab43b0280a964c18a81ab0531a0a2a10: Bootstrap starting.
I20260807 18:33:36.449395 18726 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ab43b0280a964c18a81ab0531a0a2a10: Neither blocks nor log segments found. Creating new log.
I20260807 18:33:36.450160 18726 log.cc:824] T 00000000000000000000000000000000 P ab43b0280a964c18a81ab0531a0a2a10: Log is configured to *not* fsync() on all Append() calls
I20260807 18:33:36.452210 18726 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ab43b0280a964c18a81ab0531a0a2a10: No bootstrap required, opened a new log
I20260807 18:33:36.455406 18726 raft_consensus.cc:348] T 00000000000000000000000000000000 P ab43b0280a964c18a81ab0531a0a2a10 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b176ea138c9402a95ef84541f6cc5e3" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 45409 } } peers { permanent_uuid: "1abe9cee5c844198bbc7d32fdfae4374" member_type: VOTER last_known_addr { host: "127.11.14.125" port: 39483 } } peers { permanent_uuid: "ab43b0280a964c18a81ab0531a0a2a10" member_type: VOTER last_known_addr { host: "127.11.14.124" port: 36503 } }
I20260807 18:33:36.455695 18726 raft_consensus.cc:374] T 00000000000000000000000000000000 P ab43b0280a964c18a81ab0531a0a2a10 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:33:36.455765 18726 raft_consensus.cc:729] T 00000000000000000000000000000000 P ab43b0280a964c18a81ab0531a0a2a10 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ab43b0280a964c18a81ab0531a0a2a10, State: Initialized, Role: FOLLOWER
I20260807 18:33:36.456221 18726 consensus_queue.cc:260] T 00000000000000000000000000000000 P ab43b0280a964c18a81ab0531a0a2a10 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b176ea138c9402a95ef84541f6cc5e3" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 45409 } } peers { permanent_uuid: "1abe9cee5c844198bbc7d32fdfae4374" member_type: VOTER last_known_addr { host: "127.11.14.125" port: 39483 } } peers { permanent_uuid: "ab43b0280a964c18a81ab0531a0a2a10" member_type: VOTER last_known_addr { host: "127.11.14.124" port: 36503 } }
I20260807 18:33:36.456699 18732 sys_catalog.cc:455] T 00000000000000000000000000000000 P ab43b0280a964c18a81ab0531a0a2a10 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b176ea138c9402a95ef84541f6cc5e3" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 45409 } } peers { permanent_uuid: "1abe9cee5c844198bbc7d32fdfae4374" member_type: VOTER last_known_addr { host: "127.11.14.125" port: 39483 } } peers { permanent_uuid: "ab43b0280a964c18a81ab0531a0a2a10" member_type: VOTER last_known_addr { host: "127.11.14.124" port: 36503 } } }
I20260807 18:33:36.456851 18732 sys_catalog.cc:458] T 00000000000000000000000000000000 P ab43b0280a964c18a81ab0531a0a2a10 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 18:33:36.457513 18726 sys_catalog.cc:564] T 00000000000000000000000000000000 P ab43b0280a964c18a81ab0531a0a2a10 [sys.catalog]: configured and running, proceeding with master startup.
W20260807 18:33:36.462455 18743 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ab43b0280a964c18a81ab0531a0a2a10: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 18:33:36.462554 18743 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 18:33:36.466126 18571 master_runner.cc:428] Detected that this master ab43b0280a964c18a81ab0531a0a2a10 is joining an existing cluster
I20260807 18:33:36.466179 18571 master_runner.cc:432] Initiating AddMaster RPC to add 127.11.14.124:36503
I20260807 18:33:36.469250 18732 raft_consensus.cc:482] T 00000000000000000000000000000000 P ab43b0280a964c18a81ab0531a0a2a10 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:33:36.469441 18732 raft_consensus.cc:504] T 00000000000000000000000000000000 P ab43b0280a964c18a81ab0531a0a2a10 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b176ea138c9402a95ef84541f6cc5e3" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 45409 } } peers { permanent_uuid: "1abe9cee5c844198bbc7d32fdfae4374" member_type: VOTER last_known_addr { host: "127.11.14.125" port: 39483 } } peers { permanent_uuid: "ab43b0280a964c18a81ab0531a0a2a10" member_type: VOTER last_known_addr { host: "127.11.14.124" port: 36503 } }
I20260807 18:33:36.470484 18732 leader_election.cc:288] T 00000000000000000000000000000000 P ab43b0280a964c18a81ab0531a0a2a10 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9b176ea138c9402a95ef84541f6cc5e3 (127.11.14.126:45409), 1abe9cee5c844198bbc7d32fdfae4374 (127.11.14.125:39483)
I20260807 18:33:36.471347 18611 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ab43b0280a964c18a81ab0531a0a2a10" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1abe9cee5c844198bbc7d32fdfae4374" is_pre_election: true
I20260807 18:33:36.471642 18611 raft_consensus.cc:1799] T 00000000000000000000000000000000 P 1abe9cee5c844198bbc7d32fdfae4374 [term 1 LEARNER]: Handling vote request from an unknown peer ab43b0280a964c18a81ab0531a0a2a10
I20260807 18:33:36.471803 18611 raft_consensus.cc:2396] T 00000000000000000000000000000000 P 1abe9cee5c844198bbc7d32fdfae4374 [term 1 LEARNER]: Leader pre-election vote request: Denying vote to candidate ab43b0280a964c18a81ab0531a0a2a10 for term 1 because replica has last-logged OpId of term: 1 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20260807 18:33:36.472153 17974 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ab43b0280a964c18a81ab0531a0a2a10" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b176ea138c9402a95ef84541f6cc5e3" is_pre_election: true
I20260807 18:33:36.473146 18674 leader_election.cc:302] T 00000000000000000000000000000000 P ab43b0280a964c18a81ab0531a0a2a10 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ab43b0280a964c18a81ab0531a0a2a10; no voters: 1abe9cee5c844198bbc7d32fdfae4374, 9b176ea138c9402a95ef84541f6cc5e3
I20260807 18:33:36.473625 18732 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ab43b0280a964c18a81ab0531a0a2a10 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:33:36.475075 18732 raft_consensus.cc:2726] T 00000000000000000000000000000000 P ab43b0280a964c18a81ab0531a0a2a10 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 18:33:36.478801 11321 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkz56XN/build/debug/bin/kudu with pid 18029
I20260807 18:33:36.490552 11321 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkz56XN/build/debug/bin/kudu
/tmp/dist-test-taskkz56XN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.14.65:33879
--local_ip_for_outbound_sockets=127.11.14.65
--tserver_master_addrs=127.11.14.126:45409
--webserver_port=38645
--webserver_interface=127.11.14.65
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/krb5kdc/oryx_127.11.14.65.keytab
--principal=oryx/127.11.14.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.14.84:41437
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.11.14.126:45409,127.11.14.125:39483
--tserver_master_addrs=127.11.14.126:45409,127.11.14.125:39483,127.11.14.124:36503 with env {KRB5CCNAME=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 18:33:36.496271 18751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:36.468787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.124:36503 (local address 127.0.0.1:44552)
0807 18:33:36.471033 (+ 2246us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:36.471048 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:36.471161 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:36.471426 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:36.471429 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:36.471718 (+ 289us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:33:36.471847 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:36.471856 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:36.472741 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:36.472744 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:36.473544 (+ 800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:36.473556 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:36.474994 (+ 1438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:36.474999 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:36.475100 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:36.475722 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:33:36.475734 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:33:36.476102 (+ 368us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:33:36.494108 (+ 18006us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:33:36.494113 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:33:36.494118 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:33:36.494678 (+ 560us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:33:36.495008 (+ 330us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:33:36.495014 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:33:36.495018 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:33:36.495110 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:33:36.495695 (+ 585us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:33:36.495697 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:33:36.495805 (+ 108us) client_negotiation.cc:769] Sending connection context
0807 18:33:36.495912 (+ 107us) client_negotiation.cc:240] Negotiation successful
0807 18:33:36.496057 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2168,"mutex_wait_us":1,"thread_start_us":88,"threads_started":1}
I20260807 18:33:36.496747 18728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:36.468802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44552 (local address 127.11.14.124:36503)
0807 18:33:36.468915 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:36.468919 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:36.471110 (+ 2191us) server_negotiation.cc:407] Connection header received
0807 18:33:36.471250 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:36.471253 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:36.471287 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:36.471329 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:33:36.471969 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:36.472626 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:36.473673 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:36.474029 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:36.474126 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:36.476242 (+ 2116us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:33:36.476246 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:33:36.476249 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:33:36.476286 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:33:36.493976 (+ 17690us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:36.494794 (+ 818us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:36.494800 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:36.494805 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:36.494910 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:36.495201 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:36.495206 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:36.495211 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:36.495538 (+ 327us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:33:36.495660 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:36.496486 (+ 826us) server_negotiation.cc:299] Negotiation successful
0807 18:33:36.496600 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":34,"server-negotiator.queue_time_us":52}
I20260807 18:33:36.499660 18730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:36.469120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54258 (local address 127.11.14.125:39483)
0807 18:33:36.469276 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:36.469281 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:36.469294 (+ 13us) server_negotiation.cc:407] Connection header received
0807 18:33:36.469334 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:36.469337 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:36.469385 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:36.469466 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:33:36.470291 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:36.471103 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:36.473025 (+ 1922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:36.473419 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:36.473501 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:36.493378 (+ 19877us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:33:36.493383 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:33:36.493386 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:33:36.493418 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:33:36.495145 (+ 1727us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:36.498254 (+ 3109us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:36.498258 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:36.498259 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:36.498368 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:36.498673 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:36.498679 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:36.498682 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:36.498903 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:33:36.498987 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:36.499344 (+ 357us) server_negotiation.cc:299] Negotiation successful
0807 18:33:36.499481 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260807 18:33:36.500180 18749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:36.468366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.125:39483 (local address 127.0.0.1:54258)
0807 18:33:36.468739 (+ 373us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:36.468754 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:36.468852 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:36.469708 (+ 856us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:36.469711 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:36.469996 (+ 285us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:33:36.470125 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:36.470133 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:36.471319 (+ 1186us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:36.471322 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:36.472874 (+ 1552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:36.472888 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:36.473800 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:36.473803 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:36.474305 (+ 502us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:36.492594 (+ 18289us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:33:36.492613 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:33:36.493168 (+ 555us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:33:36.495294 (+ 2126us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:33:36.495298 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:33:36.495300 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:33:36.498117 (+ 2817us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:33:36.498483 (+ 366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:33:36.498486 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:33:36.498488 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:33:36.498573 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:33:36.499045 (+ 472us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:33:36.499048 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:33:36.499134 (+ 86us) client_negotiation.cc:769] Sending connection context
0807 18:33:36.499847 (+ 713us) client_negotiation.cc:240] Negotiation successful
0807 18:33:36.499995 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":306,"mutex_wait_us":48,"thread_start_us":104,"threads_started":1}
I20260807 18:33:36.502044 18004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:36.468387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32982 (local address 127.11.14.126:45409)
0807 18:33:36.468528 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:36.468533 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:36.469029 (+ 496us) server_negotiation.cc:407] Connection header received
0807 18:33:36.469330 (+ 301us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:36.469333 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:36.469390 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:36.469447 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:33:36.470207 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:36.470969 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:36.474968 (+ 3999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:36.475748 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:36.475864 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:36.495791 (+ 19927us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:33:36.495797 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:33:36.495803 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:33:36.495853 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:33:36.498387 (+ 2534us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:36.499016 (+ 629us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:36.499020 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:36.499024 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:36.499126 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:36.499498 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:36.499505 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:36.499509 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:36.499817 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:33:36.499922 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:36.501766 (+ 1844us) server_negotiation.cc:299] Negotiation successful
0807 18:33:36.501891 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260807 18:33:36.502436 18750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:36.468344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:45409 (local address 127.0.0.1:32982)
0807 18:33:36.468695 (+ 351us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:36.468940 (+ 245us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:36.469039 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:36.469608 (+ 569us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:36.469611 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:36.469903 (+ 292us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:33:36.470060 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:36.470068 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:36.473983 (+ 3915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:36.473986 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:36.474833 (+ 847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:36.474847 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:36.475989 (+ 1142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:36.475993 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:36.476085 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:36.476528 (+ 443us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:33:36.476543 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:33:36.495583 (+ 19040us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:33:36.498562 (+ 2979us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:33:36.498568 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:33:36.498572 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:33:36.498871 (+ 299us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:33:36.499264 (+ 393us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:33:36.499270 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:33:36.499273 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:33:36.499372 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:33:36.501420 (+ 2048us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:33:36.501425 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:33:36.501545 (+ 120us) client_negotiation.cc:769] Sending connection context
0807 18:33:36.501679 (+ 134us) client_negotiation.cc:240] Negotiation successful
0807 18:33:36.501861 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"mutex_wait_us":3,"thread_start_us":117,"threads_started":1}
Aug 07 18:33:36 dist-test-slave-6k22 krb5kdc[17915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127615, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.14.126@KRBTEST.COM for oryx/127.11.14.124@KRBTEST.COM
I20260807 18:33:36.516106 18728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:36.506562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44564 (local address 127.11.14.124:36503)
0807 18:33:36.506675 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:36.506679 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:36.506778 (+ 99us) server_negotiation.cc:407] Connection header received
0807 18:33:36.506916 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:36.506919 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:36.506972 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:36.507054 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:33:36.507518 (+ 464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:36.508461 (+ 943us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:36.509633 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:36.510045 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:36.510152 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:36.512470 (+ 2318us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:33:36.512474 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:33:36.512477 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:33:36.512502 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:33:36.514200 (+ 1698us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:36.514681 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:36.514684 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:36.514686 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:36.514773 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:36.515095 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:36.515098 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:36.515100 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:36.515323 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:33:36.515421 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:36.515823 (+ 402us) server_negotiation.cc:299] Negotiation successful
0807 18:33:36.515948 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260807 18:33:36.516214 18555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:36.506464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.124:36503 (local address 127.0.0.1:44564)
0807 18:33:36.506694 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:36.506710 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:36.506798 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:36.507087 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:36.507090 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:36.507278 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:33:36.507383 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:36.507390 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:36.508575 (+ 1185us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:36.508579 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:36.509509 (+ 930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:36.509520 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:36.510158 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:36.510162 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:36.510263 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:36.510720 (+ 457us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:33:36.510734 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:33:36.512323 (+ 1589us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:33:36.514321 (+ 1998us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:33:36.514324 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:33:36.514326 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:33:36.514573 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:33:36.514891 (+ 318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:33:36.514894 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:33:36.514896 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:33:36.514992 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:33:36.515428 (+ 436us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:33:36.515433 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:33:36.515552 (+ 119us) client_negotiation.cc:769] Sending connection context
0807 18:33:36.515748 (+ 196us) client_negotiation.cc:240] Negotiation successful
0807 18:33:36.515926 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152}
I20260807 18:33:36.517511 17963 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.11.14.124:36503: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "ab43b0280a964c18a81ab0531a0a2a10"
member_type: NON_VOTER
last_known_addr {
host: "127.11.14.124"
port: 36503
}
attrs {
promote: true
}
}
dest_uuid: "9b176ea138c9402a95ef84541f6cc5e3"
cas_config_opid_index: 5
I20260807 18:33:36.518209 17963 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9b176ea138c9402a95ef84541f6cc5e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "9b176ea138c9402a95ef84541f6cc5e3" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 45409 } } peers { permanent_uuid: "1abe9cee5c844198bbc7d32fdfae4374" member_type: NON_VOTER last_known_addr { host: "127.11.14.125" port: 39483 } attrs { promote: true } } peers { permanent_uuid: "ab43b0280a964c18a81ab0531a0a2a10" member_type: NON_VOTER last_known_addr { host: "127.11.14.124" port: 36503 } attrs { promote: true } }
I20260807 18:33:36.519644 18559 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 9b176ea138c9402a95ef84541f6cc5e3 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER ab43b0280a964c18a81ab0531a0a2a10 (127.11.14.124) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "9b176ea138c9402a95ef84541f6cc5e3" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 45409 } } peers { permanent_uuid: "1abe9cee5c844198bbc7d32fdfae4374" member_type: NON_VOTER last_known_addr { host: "127.11.14.125" port: 39483 } attrs { promote: true } } peers { permanent_uuid: "ab43b0280a964c18a81ab0531a0a2a10" member_type: NON_VOTER last_known_addr { host: "127.11.14.124" port: 36503 } attrs { promote: true } } }
I20260807 18:33:36.521134 18559 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.11.14.124:36503
I20260807 18:33:36.521320 18559 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b176ea138c9402a95ef84541f6cc5e3 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "9b176ea138c9402a95ef84541f6cc5e3" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "9b176ea138c9402a95ef84541f6cc5e3" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 45409 } } peers { permanent_uuid: "1abe9cee5c844198bbc7d32fdfae4374" member_type: NON_VOTER last_known_addr { host: "127.11.14.125" port: 39483 } attrs { promote: true } } peers { permanent_uuid: "ab43b0280a964c18a81ab0531a0a2a10" member_type: NON_VOTER last_known_addr { host: "127.11.14.124" port: 36503 } attrs { promote: true } } }
I20260807 18:33:36.521418 18559 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b176ea138c9402a95ef84541f6cc5e3 [sys.catalog]: This master's current role is: LEADER
I20260807 18:33:36.521862 18571 master.cc:537] Master@127.11.14.124:36503 shutting down...
W20260807 18:33:36.523006 17946 consensus_peers.cc:487] T 00000000000000000000000000000000 P 9b176ea138c9402a95ef84541f6cc5e3 -> Peer ab43b0280a964c18a81ab0531a0a2a10 (127.11.14.124:36503): Couldn't send request to peer ab43b0280a964c18a81ab0531a0a2a10. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260807 18:33:36.527101 18571 raft_consensus.cc:2229] T 00000000000000000000000000000000 P ab43b0280a964c18a81ab0531a0a2a10 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:33:36.527460 18571 raft_consensus.cc:2258] T 00000000000000000000000000000000 P ab43b0280a964c18a81ab0531a0a2a10 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:33:36.527601 18730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:36.521581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54266 (local address 127.11.14.125:39483)
0807 18:33:36.521735 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:36.521740 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:36.522797 (+ 1057us) server_negotiation.cc:407] Connection header received
0807 18:33:36.523002 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:36.523005 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:36.523069 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:36.523135 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:33:36.523527 (+ 392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:36.524322 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:36.525853 (+ 1531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:36.526518 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:36.526600 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:36.526746 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:36.527327 (+ 581us) server_negotiation.cc:299] Negotiation successful
0807 18:33:36.527409 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260807 18:33:36.527776 18571 tablet_replica.cc:330] T 00000000000000000000000000000000 P ab43b0280a964c18a81ab0531a0a2a10: stopping tablet replica
I20260807 18:33:36.528077 18555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:36.519830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.125:39483 (local address 127.0.0.1:54266)
0807 18:33:36.522723 (+ 2893us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:36.522738 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:36.522880 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:36.523291 (+ 411us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:36.523294 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:36.523310 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:33:36.523398 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:36.523404 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:36.524475 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:36.524478 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:36.525689 (+ 1211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:36.525702 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:36.527056 (+ 1354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:36.527060 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:36.527153 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:36.527201 (+ 48us) client_negotiation.cc:769] Sending connection context
0807 18:33:36.527855 (+ 654us) client_negotiation.cc:240] Negotiation successful
0807 18:33:36.527948 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2137}
I20260807 18:33:36.528784 18611 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1abe9cee5c844198bbc7d32fdfae4374 [term 1 LEARNER]: Refusing update from remote peer 9b176ea138c9402a95ef84541f6cc5e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260807 18:33:36.529270 18753 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 9b176ea138c9402a95ef84541f6cc5e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1abe9cee5c844198bbc7d32fdfae4374" member_type: NON_VOTER last_known_addr { host: "127.11.14.125" port: 39483 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260807 18:33:36.531141 18611 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 1abe9cee5c844198bbc7d32fdfae4374 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER ab43b0280a964c18a81ab0531a0a2a10 (127.11.14.124) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "9b176ea138c9402a95ef84541f6cc5e3" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 45409 } } peers { permanent_uuid: "1abe9cee5c844198bbc7d32fdfae4374" member_type: NON_VOTER last_known_addr { host: "127.11.14.125" port: 39483 } attrs { promote: true } } peers { permanent_uuid: "ab43b0280a964c18a81ab0531a0a2a10" member_type: NON_VOTER last_known_addr { host: "127.11.14.124" port: 36503 } attrs { promote: true } } }
I20260807 18:33:36.533535 18644 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1abe9cee5c844198bbc7d32fdfae4374 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9b176ea138c9402a95ef84541f6cc5e3. Latest consensus state: current_term: 1 leader_uuid: "9b176ea138c9402a95ef84541f6cc5e3" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "9b176ea138c9402a95ef84541f6cc5e3" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 45409 } } peers { permanent_uuid: "1abe9cee5c844198bbc7d32fdfae4374" member_type: NON_VOTER last_known_addr { host: "127.11.14.125" port: 39483 } attrs { promote: true } } peers { permanent_uuid: "ab43b0280a964c18a81ab0531a0a2a10" member_type: NON_VOTER last_known_addr { host: "127.11.14.124" port: 36503 } attrs { promote: true } } }
I20260807 18:33:36.533666 18644 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1abe9cee5c844198bbc7d32fdfae4374 [sys.catalog]: This master's current role is: LEARNER
I20260807 18:33:36.533766 18644 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1abe9cee5c844198bbc7d32fdfae4374 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9b176ea138c9402a95ef84541f6cc5e3" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "9b176ea138c9402a95ef84541f6cc5e3" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 45409 } } peers { permanent_uuid: "1abe9cee5c844198bbc7d32fdfae4374" member_type: NON_VOTER last_known_addr { host: "127.11.14.125" port: 39483 } attrs { promote: true } } peers { permanent_uuid: "ab43b0280a964c18a81ab0531a0a2a10" member_type: NON_VOTER last_known_addr { host: "127.11.14.124" port: 36503 } attrs { promote: true } } }
I20260807 18:33:36.533856 18644 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1abe9cee5c844198bbc7d32fdfae4374 [sys.catalog]: This master's current role is: LEARNER
I20260807 18:33:36.534286 18571 master.cc:552] Master@127.11.14.124:36503 shutdown complete.
I20260807 18:33:36.534365 18571 master_runner.cc:305] Clearing existing system tablet
I20260807 18:33:36.534631 18571 fs_manager.cc:359] Metadata directory not provided
I20260807 18:33:36.534715 18571 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:33:36.536024 18571 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 18:33:36.536863 18753 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 9b176ea138c9402a95ef84541f6cc5e3: attempting to promote NON_VOTER 1abe9cee5c844198bbc7d32fdfae4374 to VOTER
I20260807 18:33:36.536891 18758 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:33:36.537194 18571 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:33:36.537304 18571 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 18:33:36.537336 18571 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/minicluster-data/master-2/data,/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/minicluster-data/master-2/wal
uuid: "ab43b0280a964c18a81ab0531a0a2a10"
format_stamp: "Formatted at 2026-08-07 18:33:36 on dist-test-slave-6k22"
server_key: "2d34986feb2bf6cd49bcf9ca715c6a2e"
server_key_iv: "492e11eafdd5e2f41aeea36a0e406435"
server_key_version: "encryptionkey@0"
I20260807 18:33:36.537312 18753 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9b176ea138c9402a95ef84541f6cc5e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "9b176ea138c9402a95ef84541f6cc5e3" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 45409 } } peers { permanent_uuid: "1abe9cee5c844198bbc7d32fdfae4374" member_type: VOTER last_known_addr { host: "127.11.14.125" port: 39483 } attrs { promote: false } } peers { permanent_uuid: "ab43b0280a964c18a81ab0531a0a2a10" member_type: NON_VOTER last_known_addr { host: "127.11.14.124" port: 36503 } attrs { promote: true } }
I20260807 18:33:36.539028 18555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:36.538645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.124:36503 (local address 127.0.0.1:44576)
0807 18:33:36.538878 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:36.538953 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.124:36503: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":152}
I20260807 18:33:36.539398 18611 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1abe9cee5c844198bbc7d32fdfae4374 [term 1 LEARNER]: Refusing update from remote peer 9b176ea138c9402a95ef84541f6cc5e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260807 18:33:36.539391 18571 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P ab43b0280a964c18a81ab0531a0a2a10: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:33:36.540068 18559 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 9b176ea138c9402a95ef84541f6cc5e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1abe9cee5c844198bbc7d32fdfae4374" member_type: VOTER last_known_addr { host: "127.11.14.125" port: 39483 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260807 18:33:36.540799 18555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:36.540439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.124:36503 (local address 127.0.0.1:44584)
0807 18:33:36.540661 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:36.540721 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.124:36503: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146}
W20260807 18:33:36.541566 17946 consensus_peers.cc:487] T 00000000000000000000000000000000 P 9b176ea138c9402a95ef84541f6cc5e3 -> Peer ab43b0280a964c18a81ab0531a0a2a10 (127.11.14.124:36503): Couldn't send request to peer ab43b0280a964c18a81ab0531a0a2a10. Status: Network error: Client connection negotiation failed: client connection to 127.11.14.124:36503: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260807 18:33:36.542408 18571 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P ab43b0280a964c18a81ab0531a0a2a10: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260807 18:33:36.542542 18571 log.cc:1196] T 00000000000000000000000000000000 P ab43b0280a964c18a81ab0531a0a2a10: Deleting WAL directory at /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260807 18:33:36.542490 18559 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 9b176ea138c9402a95ef84541f6cc5e3 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 1abe9cee5c844198bbc7d32fdfae4374 (127.11.14.125) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "9b176ea138c9402a95ef84541f6cc5e3" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 45409 } } peers { permanent_uuid: "1abe9cee5c844198bbc7d32fdfae4374" member_type: VOTER last_known_addr { host: "127.11.14.125" port: 39483 } attrs { promote: false } } peers { permanent_uuid: "ab43b0280a964c18a81ab0531a0a2a10" member_type: NON_VOTER last_known_addr { host: "127.11.14.124" port: 36503 } attrs { promote: true } } }
I20260807 18:33:36.542968 18571 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P ab43b0280a964c18a81ab0531a0a2a10: Deleting consensus metadata
I20260807 18:33:36.543205 18571 master_runner.cc:315] Copying system tablet from 127.11.14.126:45409
I20260807 18:33:36.543967 18559 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b176ea138c9402a95ef84541f6cc5e3 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "9b176ea138c9402a95ef84541f6cc5e3" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "9b176ea138c9402a95ef84541f6cc5e3" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 45409 } } peers { permanent_uuid: "1abe9cee5c844198bbc7d32fdfae4374" member_type: VOTER last_known_addr { host: "127.11.14.125" port: 39483 } attrs { promote: false } } peers { permanent_uuid: "ab43b0280a964c18a81ab0531a0a2a10" member_type: NON_VOTER last_known_addr { host: "127.11.14.124" port: 36503 } attrs { promote: true } } }
I20260807 18:33:36.544075 18559 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b176ea138c9402a95ef84541f6cc5e3 [sys.catalog]: This master's current role is: LEADER
I20260807 18:33:36.544199 18559 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b176ea138c9402a95ef84541f6cc5e3 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9b176ea138c9402a95ef84541f6cc5e3" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "9b176ea138c9402a95ef84541f6cc5e3" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 45409 } } peers { permanent_uuid: "1abe9cee5c844198bbc7d32fdfae4374" member_type: VOTER last_known_addr { host: "127.11.14.125" port: 39483 } attrs { promote: false } } peers { permanent_uuid: "ab43b0280a964c18a81ab0531a0a2a10" member_type: NON_VOTER last_known_addr { host: "127.11.14.124" port: 36503 } attrs { promote: true } } }
I20260807 18:33:36.544275 18559 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b176ea138c9402a95ef84541f6cc5e3 [sys.catalog]: This master's current role is: LEADER
I20260807 18:33:36.544643 18571 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P ab43b0280a964c18a81ab0531a0a2a10: tablet copy: Beginning tablet copy session from remote peer at address 127.11.14.126:45409
I20260807 18:33:36.544955 18611 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 1abe9cee5c844198bbc7d32fdfae4374 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 1abe9cee5c844198bbc7d32fdfae4374 (127.11.14.125) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "9b176ea138c9402a95ef84541f6cc5e3" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 45409 } } peers { permanent_uuid: "1abe9cee5c844198bbc7d32fdfae4374" member_type: VOTER last_known_addr { host: "127.11.14.125" port: 39483 } attrs { promote: false } } peers { permanent_uuid: "ab43b0280a964c18a81ab0531a0a2a10" member_type: NON_VOTER last_known_addr { host: "127.11.14.124" port: 36503 } attrs { promote: true } } }
I20260807 18:33:36.546753 18755 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1abe9cee5c844198bbc7d32fdfae4374 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9b176ea138c9402a95ef84541f6cc5e3" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "9b176ea138c9402a95ef84541f6cc5e3" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 45409 } } peers { permanent_uuid: "1abe9cee5c844198bbc7d32fdfae4374" member_type: VOTER last_known_addr { host: "127.11.14.125" port: 39483 } attrs { promote: false } } peers { permanent_uuid: "ab43b0280a964c18a81ab0531a0a2a10" member_type: NON_VOTER last_known_addr { host: "127.11.14.124" port: 36503 } attrs { promote: true } } }
I20260807 18:33:36.546895 18755 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1abe9cee5c844198bbc7d32fdfae4374 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 18:33:36.555054 18004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:36.545440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32992 (local address 127.11.14.126:45409)
0807 18:33:36.545539 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:36.545543 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:36.546439 (+ 896us) server_negotiation.cc:407] Connection header received
0807 18:33:36.546619 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:36.546622 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:36.546668 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:36.546721 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:33:36.547435 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:36.548273 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:36.549542 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:36.549872 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:36.549943 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:36.551406 (+ 1463us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:33:36.551410 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:33:36.551413 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:33:36.551437 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:33:36.553193 (+ 1756us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:36.553708 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:36.553712 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:36.553714 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:36.553805 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:36.554121 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:36.554125 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:36.554127 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:36.554348 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:33:36.554488 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:36.554786 (+ 298us) server_negotiation.cc:299] Negotiation successful
0807 18:33:36.554904 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260807 18:33:36.555071 18765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:36.545623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:45409 (local address 127.0.0.1:32992)
0807 18:33:36.546357 (+ 734us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:36.546371 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:36.546514 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:36.546862 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:36.546865 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:36.547179 (+ 314us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:33:36.547301 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:36.547308 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:36.548415 (+ 1107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:36.548418 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:36.549395 (+ 977us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:36.549405 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:36.550102 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:36.550106 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:36.550208 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:36.550786 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:33:36.550801 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:33:36.551250 (+ 449us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:33:36.553320 (+ 2070us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:33:36.553323 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:33:36.553325 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:33:36.553592 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:33:36.553912 (+ 320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:33:36.553916 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:33:36.553918 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:33:36.554016 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:33:36.554487 (+ 471us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:33:36.554491 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:33:36.554598 (+ 107us) client_negotiation.cc:769] Sending connection context
0807 18:33:36.554713 (+ 115us) client_negotiation.cc:240] Negotiation successful
0807 18:33:36.554866 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":666,"thread_start_us":59,"threads_started":1}
I20260807 18:33:36.555814 17984 tablet_copy_service.cc:140] P 9b176ea138c9402a95ef84541f6cc5e3: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer ab43b0280a964c18a81ab0531a0a2a10 ({username='oryx', principal='oryx/127.11.14.124@KRBTEST.COM'} at 127.0.0.1:32992)
I20260807 18:33:36.555908 17984 tablet_copy_service.cc:161] P 9b176ea138c9402a95ef84541f6cc5e3: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer ab43b0280a964c18a81ab0531a0a2a10 at {username='oryx', principal='oryx/127.11.14.124@KRBTEST.COM'} at 127.0.0.1:32992: session id = ab43b0280a964c18a81ab0531a0a2a10-00000000000000000000000000000000
I20260807 18:33:36.556979 17984 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 9b176ea138c9402a95ef84541f6cc5e3: Tablet Copy: opened 0 blocks and 1 log segments
I20260807 18:33:36.558327 18571 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260807 18:33:36.560842 18571 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P ab43b0280a964c18a81ab0531a0a2a10: tablet copy: Starting download of 0 data blocks...
I20260807 18:33:36.561035 18571 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P ab43b0280a964c18a81ab0531a0a2a10: tablet copy: Starting download of 1 WAL segments...
I20260807 18:33:36.562513 18571 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P ab43b0280a964c18a81ab0531a0a2a10: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260807 18:33:36.564260 17984 tablet_copy_service.cc:342] P 9b176ea138c9402a95ef84541f6cc5e3: Request end of tablet copy session ab43b0280a964c18a81ab0531a0a2a10-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.11.14.124@KRBTEST.COM'} at 127.0.0.1:32992
I20260807 18:33:36.564340 17984 tablet_copy_service.cc:433] P 9b176ea138c9402a95ef84541f6cc5e3: ending tablet copy session ab43b0280a964c18a81ab0531a0a2a10-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer ab43b0280a964c18a81ab0531a0a2a10
I20260807 18:33:36.571967 18571 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:33:36.573659 18766 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:33:36.576669 18767 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:33:36.576841 18770 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:33:36.578913 18571 server_base.cc:1034] running on GCE node
I20260807 18:33:36.579075 18571 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:33:36.579370 18571 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:33:36.580482 18571 hybrid_clock.cc:648] HybridClock initialized: now 1786127616580435 us; error 62 us; skew 500 ppm
Aug 07 18:33:36 dist-test-slave-6k22 krb5kdc[17915](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127616, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.14.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:33:36.582219 18571 init.cc:377] Logged in from keytab as oryx/127.11.14.124@KRBTEST.COM (short username oryx)
I20260807 18:33:36.582687 18571 webserver.cc:460] Webserver started at http://127.11.14.124:36021/ using document root <none> and password file <none>
I20260807 18:33:36.582882 18571 fs_manager.cc:359] Metadata directory not provided
I20260807 18:33:36.582937 18571 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:33:36.584506 18571 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 18:33:36.591727 18778 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:33:36.592006 18571 fs_manager.cc:725] Time spent opening block manager: real 0.007s user 0.001s sys 0.000s
I20260807 18:33:36.592087 18571 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/minicluster-data/master-2/data,/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/minicluster-data/master-2/wal
uuid: "ab43b0280a964c18a81ab0531a0a2a10"
format_stamp: "Formatted at 2026-08-07 18:33:36 on dist-test-slave-6k22"
server_key: "2d34986feb2bf6cd49bcf9ca715c6a2e"
server_key_iv: "492e11eafdd5e2f41aeea36a0e406435"
server_key_version: "encryptionkey@0"
I20260807 18:33:36.592172 18571 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:33:36.599181 18571 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:33:36.601475 18571 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:33:36.606648 18571 rpc_server.cc:307] RPC server started. Bound to: 127.11.14.124:36503
I20260807 18:33:36.606755 18834 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.14.124:36503 every 8 connection(s)
I20260807 18:33:36.607257 18571 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/minicluster-data/master-2/data/info.pb
I20260807 18:33:36.609362 18835 sys_catalog.cc:263] Verifying existing consensus state
I20260807 18:33:36.610548 18835 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ab43b0280a964c18a81ab0531a0a2a10: Bootstrap starting.
I20260807 18:33:36.610724 18306 heartbeater.cc:503] Master 127.11.14.126:45409 was elected leader, sending a full tablet report...
I20260807 18:33:36.622895 18835 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ab43b0280a964c18a81ab0531a0a2a10: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 18:33:36.623217 18835 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ab43b0280a964c18a81ab0531a0a2a10: Bootstrap complete.
I20260807 18:33:36.623833 18835 raft_consensus.cc:348] T 00000000000000000000000000000000 P ab43b0280a964c18a81ab0531a0a2a10 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "9b176ea138c9402a95ef84541f6cc5e3" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 45409 } } peers { permanent_uuid: "1abe9cee5c844198bbc7d32fdfae4374" member_type: VOTER last_known_addr { host: "127.11.14.125" port: 39483 } attrs { promote: false } } peers { permanent_uuid: "ab43b0280a964c18a81ab0531a0a2a10" member_type: NON_VOTER last_known_addr { host: "127.11.14.124" port: 36503 } attrs { promote: true } }
I20260807 18:33:36.623984 18835 raft_consensus.cc:729] T 00000000000000000000000000000000 P ab43b0280a964c18a81ab0531a0a2a10 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: ab43b0280a964c18a81ab0531a0a2a10, State: Initialized, Role: LEARNER
I20260807 18:33:36.624096 18835 consensus_queue.cc:260] T 00000000000000000000000000000000 P ab43b0280a964c18a81ab0531a0a2a10 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "9b176ea138c9402a95ef84541f6cc5e3" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 45409 } } peers { permanent_uuid: "1abe9cee5c844198bbc7d32fdfae4374" member_type: VOTER last_known_addr { host: "127.11.14.125" port: 39483 } attrs { promote: false } } peers { permanent_uuid: "ab43b0280a964c18a81ab0531a0a2a10" member_type: NON_VOTER last_known_addr { host: "127.11.14.124" port: 36503 } attrs { promote: true } }
I20260807 18:33:36.624625 18835 sys_catalog.cc:564] T 00000000000000000000000000000000 P ab43b0280a964c18a81ab0531a0a2a10 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:33:36.624588 18838 sys_catalog.cc:455] T 00000000000000000000000000000000 P ab43b0280a964c18a81ab0531a0a2a10 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "9b176ea138c9402a95ef84541f6cc5e3" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 45409 } } peers { permanent_uuid: "1abe9cee5c844198bbc7d32fdfae4374" member_type: VOTER last_known_addr { host: "127.11.14.125" port: 39483 } attrs { promote: false } } peers { permanent_uuid: "ab43b0280a964c18a81ab0531a0a2a10" member_type: NON_VOTER last_known_addr { host: "127.11.14.124" port: 36503 } attrs { promote: true } } }
I20260807 18:33:36.624833 18838 sys_catalog.cc:458] T 00000000000000000000000000000000 P ab43b0280a964c18a81ab0531a0a2a10 [sys.catalog]: This master's current role is: LEARNER
I20260807 18:33:36.628777 18851 catalog_manager.cc:1259] Loaded cluster ID: b84b754b281a4b06b86e0ce85d0fbbe8
I20260807 18:33:36.628834 18851 catalog_manager.cc:1552] T 00000000000000000000000000000000 P ab43b0280a964c18a81ab0531a0a2a10: loading cluster ID for follower catalog manager: success
I20260807 18:33:36.631203 18851 catalog_manager.cc:1574] T 00000000000000000000000000000000 P ab43b0280a964c18a81ab0531a0a2a10: acquiring CA information for follower catalog manager: success
I20260807 18:33:36.632942 18851 catalog_manager.cc:1602] T 00000000000000000000000000000000 P ab43b0280a964c18a81ab0531a0a2a10: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260807 18:33:36.642964 18752 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:33:36.643244 18752 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:33:36.643334 18752 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:33:36.648005 18752 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:33:36.648072 18752 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:33:36.648182 18752 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.14.65
I20260807 18:33:36.654227 18752 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.14.84:41437
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/krb5kdc/oryx_127.11.14.65.keytab
--principal=oryx/127.11.14.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.14.65:33879
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.14.65
--webserver_port=38645
--tserver_master_addrs=127.11.14.126:45409,127.11.14.125:39483,127.11.14.124:36503
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18752
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.14.65
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:33:36.655483 18752 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:33:36.656785 18752 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:33:36.667786 18857 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:33:36.667778 18859 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:33:36.667838 18752 server_base.cc:1034] running on GCE node
W20260807 18:33:36.667778 18856 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:33:36.668453 18752 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:33:36.669056 18752 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:33:36.670224 18752 hybrid_clock.cc:648] HybridClock initialized: now 1786127616670205 us; error 28 us; skew 500 ppm
Aug 07 18:33:36 dist-test-slave-6k22 krb5kdc[17915](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127616, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.14.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:33:36.673566 18752 init.cc:377] Logged in from keytab as oryx/127.11.14.65@KRBTEST.COM (short username oryx)
I20260807 18:33:36.674448 18752 webserver.cc:460] Webserver started at http://127.11.14.65:38645/ using document root <none> and password file <none>
I20260807 18:33:36.675050 18752 fs_manager.cc:359] Metadata directory not provided
I20260807 18:33:36.675168 18752 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:33:36.679302 18752 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:33:36.681722 18867 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:33:36.683143 18752 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 18:33:36.683310 18752 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/minicluster-data/ts-0/data,/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/minicluster-data/ts-0/wal
uuid: "a290fe9b958d48f0a540e728e58c4bc7"
format_stamp: "Formatted at 2026-08-07 18:33:35 on dist-test-slave-6k22"
server_key: "005363606f383b886e3255a05d63e888"
server_key_iv: "9e4faa765082d77eb388fab665733aca"
server_key_version: "encryptionkey@0"
I20260807 18:33:36.683995 18752 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:33:36.714818 18752 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:33:36.718641 18752 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:33:36.718874 18752 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:33:36.719551 18752 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:33:36.720587 18752 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:33:36.720664 18752 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:33:36.720748 18752 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:33:36.720791 18752 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:33:36.731451 18752 rpc_server.cc:307] RPC server started. Bound to: 127.11.14.65:33879
I20260807 18:33:36.731521 18990 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.14.65:33879 every 8 connection(s)
I20260807 18:33:36.732640 18752 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/minicluster-data/ts-0/data/info.pb
I20260807 18:33:36.741341 11321 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkz56XN/build/debug/bin/kudu as pid 18752
I20260807 18:33:36.741569 11321 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkz56XN/build/debug/bin/kudu with pid 18170
Aug 07 18:33:36 dist-test-slave-6k22 krb5kdc[17915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127616, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.14.65@KRBTEST.COM for oryx/127.11.14.124@KRBTEST.COM
Aug 07 18:33:36 dist-test-slave-6k22 krb5kdc[17915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127616, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.14.65@KRBTEST.COM for oryx/127.11.14.126@KRBTEST.COM
I20260807 18:33:36.750469 18995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:36.734564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.65:41085 (local address 127.11.14.124:36503)
0807 18:33:36.734933 (+ 369us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:36.734952 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:36.735758 (+ 806us) server_negotiation.cc:407] Connection header received
0807 18:33:36.737069 (+ 1311us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:36.737076 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:36.737174 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:36.737262 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:33:36.738826 (+ 1564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:36.740067 (+ 1241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:36.741303 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:36.741663 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:36.741781 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:36.745434 (+ 3653us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:33:36.745438 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:33:36.745441 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:33:36.745472 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:33:36.747544 (+ 2072us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:36.748354 (+ 810us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:36.748359 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:36.748361 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:36.748457 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:36.748940 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:36.748943 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:36.748945 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:36.749222 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:33:36.749309 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:36.750108 (+ 799us) server_negotiation.cc:299] Negotiation successful
0807 18:33:36.750267 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":259,"thread_start_us":99,"threads_started":1}
I20260807 18:33:36.751299 18004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:36.734569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.65:38963 (local address 127.11.14.126:45409)
0807 18:33:36.734713 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:36.734719 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:36.735757 (+ 1038us) server_negotiation.cc:407] Connection header received
0807 18:33:36.736970 (+ 1213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:36.736974 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:36.737043 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:36.737146 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:33:36.738804 (+ 1658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:36.739359 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:36.740747 (+ 1388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:36.740987 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:36.741115 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:36.746257 (+ 5142us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:33:36.746264 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:33:36.746271 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:33:36.746310 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:33:36.748804 (+ 2494us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:36.749508 (+ 704us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:36.749512 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:36.749514 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:36.749606 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:36.749942 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:36.749946 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:36.749947 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:36.750204 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:33:36.750298 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:36.750689 (+ 391us) server_negotiation.cc:299] Negotiation successful
0807 18:33:36.750813 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
Aug 07 18:33:36 dist-test-slave-6k22 krb5kdc[17915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127616, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.14.65@KRBTEST.COM for oryx/127.11.14.125@KRBTEST.COM
I20260807 18:33:36.752844 18997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:36.735077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.124:36503 (local address 127.11.14.65:41085)
0807 18:33:36.735593 (+ 516us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:36.735650 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:36.736737 (+ 1087us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:36.737391 (+ 654us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:36.737393 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:36.737954 (+ 561us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:33:36.738647 (+ 693us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:36.738666 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:36.740218 (+ 1552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:36.740221 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:36.741176 (+ 955us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:36.741184 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:36.741924 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:36.741930 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:36.742070 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:36.742751 (+ 681us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:33:36.742770 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:33:36.745232 (+ 2462us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:33:36.747759 (+ 2527us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:33:36.747767 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:33:36.747785 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:33:36.748213 (+ 428us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:33:36.748597 (+ 384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:33:36.748602 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:33:36.748605 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:33:36.748814 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:33:36.749403 (+ 589us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:33:36.749413 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:33:36.749795 (+ 382us) client_negotiation.cc:769] Sending connection context
0807 18:33:36.750944 (+ 1149us) client_negotiation.cc:240] Negotiation successful
0807 18:33:36.751599 (+ 655us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":379,"mutex_wait_us":197,"spinlock_wait_cycles":9984,"thread_start_us":178,"threads_started":1}
I20260807 18:33:36.752844 18996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:36.735051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:45409 (local address 127.11.14.65:38963)
0807 18:33:36.735592 (+ 541us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:36.735650 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:36.736737 (+ 1087us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:36.737333 (+ 596us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:36.737343 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:36.737991 (+ 648us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:33:36.738646 (+ 655us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:36.738662 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:36.739512 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:36.739515 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:36.740613 (+ 1098us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:36.740624 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:36.741305 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:36.741309 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:36.741546 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:36.742578 (+ 1032us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:33:36.742606 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:33:36.745684 (+ 3078us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:33:36.749106 (+ 3422us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:33:36.749108 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:33:36.749110 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:33:36.749388 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:33:36.749727 (+ 339us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:33:36.749731 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:33:36.749732 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:33:36.749821 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:33:36.750366 (+ 545us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:33:36.750369 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:33:36.750461 (+ 92us) client_negotiation.cc:769] Sending connection context
0807 18:33:36.751540 (+ 1079us) client_negotiation.cc:240] Negotiation successful
0807 18:33:36.751722 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":310,"mutex_wait_us":86,"spinlock_wait_cycles":38784,"thread_start_us":115,"threads_started":1}
I20260807 18:33:36.754741 18992 heartbeater.cc:347] Connected to a master server at 127.11.14.126:45409
I20260807 18:33:36.755084 18992 heartbeater.cc:464] Registering TS with master...
I20260807 18:33:36.755976 18992 heartbeater.cc:511] Master 127.11.14.126:45409 requested a full tablet report, sending...
I20260807 18:33:36.757470 17963 ts_manager.cc:194] Re-registered known tserver with Master: a290fe9b958d48f0a540e728e58c4bc7 (127.11.14.65:33879)
I20260807 18:33:36.758399 17963 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.11.14.65@KRBTEST.COM'} at 127.11.14.65:38963
I20260807 18:33:36.760217 18730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:36.736397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.65:41803 (local address 127.11.14.125:39483)
0807 18:33:36.736561 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:36.736564 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:36.736578 (+ 14us) server_negotiation.cc:407] Connection header received
0807 18:33:36.736979 (+ 401us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:36.736982 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:36.737041 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:36.737123 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:33:36.738825 (+ 1702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:36.739747 (+ 922us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:36.749663 (+ 9916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:36.750201 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:36.750368 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:36.753604 (+ 3236us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:33:36.753612 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:33:36.753618 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:33:36.753661 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:33:36.756321 (+ 2660us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:36.758451 (+ 2130us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:36.758454 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:36.758455 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:36.758606 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:36.758929 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:36.758933 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:36.758935 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:36.759209 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:33:36.759316 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:36.759775 (+ 459us) server_negotiation.cc:299] Negotiation successful
0807 18:33:36.759950 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260807 18:33:36.760428 18991 heartbeater.cc:347] Connected to a master server at 127.11.14.124:36503
I20260807 18:33:36.760520 18991 heartbeater.cc:464] Registering TS with master...
I20260807 18:33:36.760927 18991 heartbeater.cc:511] Master 127.11.14.124:36503 requested a full tablet report, sending...
I20260807 18:33:36.760934 18998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:36.736106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.125:39483 (local address 127.11.14.65:41803)
0807 18:33:36.736363 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:36.736375 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:36.736737 (+ 362us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:36.737342 (+ 605us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:36.737345 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:36.737973 (+ 628us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:33:36.738647 (+ 674us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:36.738666 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:36.739898 (+ 1232us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:36.739901 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:36.740953 (+ 1052us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:36.740964 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:36.750558 (+ 9594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:36.750567 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:36.750722 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:36.751242 (+ 520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:33:36.751264 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:33:36.753381 (+ 2117us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:33:36.756449 (+ 3068us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:33:36.756456 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:33:36.756459 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:33:36.758253 (+ 1794us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:33:36.758723 (+ 470us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:33:36.758729 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:33:36.758731 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:33:36.758837 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:33:36.759370 (+ 533us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:33:36.759377 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:33:36.759531 (+ 154us) client_negotiation.cc:769] Sending connection context
0807 18:33:36.760438 (+ 907us) client_negotiation.cc:240] Negotiation successful
0807 18:33:36.760637 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":202,"mutex_wait_us":72,"spinlock_wait_cycles":18048,"thread_start_us":85,"threads_started":1}
I20260807 18:33:36.762017 18993 heartbeater.cc:347] Connected to a master server at 127.11.14.125:39483
I20260807 18:33:36.762128 18993 heartbeater.cc:464] Registering TS with master...
I20260807 18:33:36.762296 18993 heartbeater.cc:511] Master 127.11.14.125:39483 requested a full tablet report, sending...
I20260807 18:33:36.762419 18796 ts_manager.cc:194] Registered new tserver with Master: a290fe9b958d48f0a540e728e58c4bc7 (127.11.14.65:33879)
I20260807 18:33:36.763509 18600 ts_manager.cc:194] Registered new tserver with Master: a290fe9b958d48f0a540e728e58c4bc7 (127.11.14.65:33879)
I20260807 18:33:36.765196 11321 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkz56XN/build/debug/bin/kudu
/tmp/dist-test-taskkz56XN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.14.66:39193
--local_ip_for_outbound_sockets=127.11.14.66
--tserver_master_addrs=127.11.14.126:45409
--webserver_port=35379
--webserver_interface=127.11.14.66
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/krb5kdc/oryx_127.11.14.66.keytab
--principal=oryx/127.11.14.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.14.84:41437
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.11.14.126:45409,127.11.14.125:39483
--tserver_master_addrs=127.11.14.126:45409,127.11.14.125:39483,127.11.14.124:36503 with env {KRB5CCNAME=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 18:33:36.831286 18446 heartbeater.cc:503] Master 127.11.14.126:45409 was elected leader, sending a full tablet report...
W20260807 18:33:36.885073 18999 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:33:36.885408 18999 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:33:36.885497 18999 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:33:36.889775 18999 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:33:36.889858 18999 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:33:36.889953 18999 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.14.66
I20260807 18:33:36.895073 18999 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.14.84:41437
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/krb5kdc/oryx_127.11.14.66.keytab
--principal=oryx/127.11.14.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.14.66:39193
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.14.66
--webserver_port=35379
--tserver_master_addrs=127.11.14.126:45409,127.11.14.125:39483,127.11.14.124:36503
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18999
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.14.66
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:33:36.896179 18999 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:33:36.897225 18999 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:33:36.907310 19005 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:33:36.907538 18999 server_base.cc:1034] running on GCE node
W20260807 18:33:36.907341 19004 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:33:36.907341 19007 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:33:36.908234 18999 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:33:36.909003 18999 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:33:36.910202 18999 hybrid_clock.cc:648] HybridClock initialized: now 1786127616910181 us; error 31 us; skew 500 ppm
Aug 07 18:33:36 dist-test-slave-6k22 krb5kdc[17915](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127616, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.14.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:33:36.913744 18999 init.cc:377] Logged in from keytab as oryx/127.11.14.66@KRBTEST.COM (short username oryx)
I20260807 18:33:36.914752 18999 webserver.cc:460] Webserver started at http://127.11.14.66:35379/ using document root <none> and password file <none>
I20260807 18:33:36.915403 18999 fs_manager.cc:359] Metadata directory not provided
I20260807 18:33:36.915520 18999 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:33:36.919739 18999 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.003s
I20260807 18:33:36.921970 19014 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:33:36.923405 18999 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 18:33:36.923550 18999 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/minicluster-data/ts-1/data,/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/minicluster-data/ts-1/wal
uuid: "f25e947f8c2c4fef895cfd78048211d7"
format_stamp: "Formatted at 2026-08-07 18:33:35 on dist-test-slave-6k22"
server_key: "7eddcc89955fb4bed7d540a39930cd33"
server_key_iv: "dea686619bd9834f8f96201649be624f"
server_key_version: "encryptionkey@0"
I20260807 18:33:36.924294 18999 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:33:36.947333 18999 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:33:36.950879 18999 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:33:36.951119 18999 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:33:36.951922 18999 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:33:36.953143 18999 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:33:36.953236 18999 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:33:36.953317 18999 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:33:36.953353 18999 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:33:36.963860 18999 rpc_server.cc:307] RPC server started. Bound to: 127.11.14.66:39193
I20260807 18:33:36.963893 19137 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.14.66:39193 every 8 connection(s)
I20260807 18:33:36.965024 18999 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/minicluster-data/ts-1/data/info.pb
I20260807 18:33:36.972868 11321 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkz56XN/build/debug/bin/kudu as pid 18999
I20260807 18:33:36.973071 11321 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkz56XN/build/debug/bin/kudu with pid 18311
Aug 07 18:33:36 dist-test-slave-6k22 krb5kdc[17915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127616, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.14.66@KRBTEST.COM for oryx/127.11.14.125@KRBTEST.COM
I20260807 18:33:36.985664 18730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:36.966986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.66:36741 (local address 127.11.14.125:39483)
0807 18:33:36.967204 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:36.967211 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:36.968185 (+ 974us) server_negotiation.cc:407] Connection header received
0807 18:33:36.969727 (+ 1542us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:36.969730 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:36.969776 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:36.969879 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:33:36.971155 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:36.971801 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:36.977491 (+ 5690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:36.977748 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:36.977877 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:36.980996 (+ 3119us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:33:36.981003 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:33:36.981009 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:33:36.981045 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:33:36.982889 (+ 1844us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:36.983636 (+ 747us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:36.983641 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:36.983644 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:36.983769 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:36.984228 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:36.984232 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:36.984233 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:36.984568 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:33:36.984684 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:36.985381 (+ 697us) server_negotiation.cc:299] Negotiation successful
0807 18:33:36.985509 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260807 18:33:36.986891 19144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:36.967471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.125:39483 (local address 127.11.14.66:36741)
0807 18:33:36.968021 (+ 550us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:36.968079 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:36.969206 (+ 1127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:36.969901 (+ 695us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:36.969905 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:36.970401 (+ 496us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:33:36.970981 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:36.970996 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:36.971961 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:36.971965 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:36.977329 (+ 5364us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:36.977341 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:36.977926 (+ 585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:36.977931 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:36.978139 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:36.978836 (+ 697us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:33:36.978856 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:33:36.980792 (+ 1936us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:33:36.983044 (+ 2252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:33:36.983053 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:33:36.983072 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:33:36.983475 (+ 403us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:33:36.983887 (+ 412us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:33:36.983893 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:33:36.983898 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:33:36.984092 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:33:36.984706 (+ 614us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:33:36.984716 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:33:36.985055 (+ 339us) client_negotiation.cc:769] Sending connection context
0807 18:33:36.985288 (+ 233us) client_negotiation.cc:240] Negotiation successful
0807 18:33:36.985662 (+ 374us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":383,"mutex_wait_us":31,"spinlock_wait_cycles":4096,"thread_start_us":153,"threads_started":1}
Aug 07 18:33:36 dist-test-slave-6k22 krb5kdc[17915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127616, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.14.66@KRBTEST.COM for oryx/127.11.14.124@KRBTEST.COM
I20260807 18:33:36.989171 19140 heartbeater.cc:347] Connected to a master server at 127.11.14.125:39483
I20260807 18:33:36.989431 19140 heartbeater.cc:464] Registering TS with master...
Aug 07 18:33:36 dist-test-slave-6k22 krb5kdc[17915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127616, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.14.66@KRBTEST.COM for oryx/127.11.14.126@KRBTEST.COM
I20260807 18:33:36.990844 19140 heartbeater.cc:511] Master 127.11.14.125:39483 requested a full tablet report, sending...
I20260807 18:33:36.992175 18600 ts_manager.cc:194] Registered new tserver with Master: f25e947f8c2c4fef895cfd78048211d7 (127.11.14.66:39193)
I20260807 18:33:36.994192 18995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:36.968751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.66:60287 (local address 127.11.14.124:36503)
0807 18:33:36.968899 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:36.968904 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:36.968960 (+ 56us) server_negotiation.cc:407] Connection header received
0807 18:33:36.969454 (+ 494us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:36.969459 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:36.969526 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:36.969595 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:33:36.971156 (+ 1561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:36.971998 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:36.985744 (+ 13746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:36.986139 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:36.986234 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:36.989397 (+ 3163us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:33:36.989401 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:33:36.989404 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:33:36.989435 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:33:36.991980 (+ 2545us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:36.992700 (+ 720us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:36.992705 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:36.992708 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:36.992808 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:36.993147 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:36.993151 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:36.993153 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:36.993425 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:33:36.993530 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:36.993852 (+ 322us) server_negotiation.cc:299] Negotiation successful
0807 18:33:36.994015 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260807 18:33:36.994191 19145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:36.968586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.124:36503 (local address 127.11.14.66:60287)
0807 18:33:36.968884 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:36.968899 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:36.969221 (+ 322us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:36.969812 (+ 591us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:36.969820 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:36.970418 (+ 598us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:33:36.971000 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:36.971010 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:36.972151 (+ 1141us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:36.972155 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:36.985546 (+ 13391us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:36.985557 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:36.986421 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:36.986425 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:36.986534 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:36.987412 (+ 878us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:33:36.987428 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:33:36.989205 (+ 1777us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:33:36.992131 (+ 2926us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:33:36.992135 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:33:36.992137 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:33:36.992496 (+ 359us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:33:36.992937 (+ 441us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:33:36.992940 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:33:36.992942 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:33:36.993030 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:33:36.993529 (+ 499us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:33:36.993533 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:33:36.993656 (+ 123us) client_negotiation.cc:769] Sending connection context
0807 18:33:36.993774 (+ 118us) client_negotiation.cc:240] Negotiation successful
0807 18:33:36.993934 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":218,"mutex_wait_us":232,"spinlock_wait_cycles":24960,"thread_start_us":97,"threads_started":1}
I20260807 18:33:36.994285 11321 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkz56XN/build/debug/bin/kudu
/tmp/dist-test-taskkz56XN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.14.67:42159
--local_ip_for_outbound_sockets=127.11.14.67
--tserver_master_addrs=127.11.14.126:45409
--webserver_port=40269
--webserver_interface=127.11.14.67
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/krb5kdc/oryx_127.11.14.67.keytab
--principal=oryx/127.11.14.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.14.84:41437
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.11.14.126:45409,127.11.14.125:39483
--tserver_master_addrs=127.11.14.126:45409,127.11.14.125:39483,127.11.14.124:36503 with env {KRB5CCNAME=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 18:33:36.994747 18004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:36.967133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.66:52085 (local address 127.11.14.126:45409)
0807 18:33:36.967256 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:36.967261 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:36.968179 (+ 918us) server_negotiation.cc:407] Connection header received
0807 18:33:36.969442 (+ 1263us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:36.969445 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:36.969514 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:36.969593 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:33:36.971155 (+ 1562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:36.971999 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:36.987366 (+ 15367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:36.987744 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:36.987887 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:36.990400 (+ 2513us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:33:36.990405 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:33:36.990408 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:33:36.990446 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:33:36.992644 (+ 2198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:36.993207 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:36.993211 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:36.993213 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:36.993300 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:36.993660 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:36.993663 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:36.993666 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:36.993901 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:33:36.994002 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:36.994438 (+ 436us) server_negotiation.cc:299] Negotiation successful
0807 18:33:36.994590 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260807 18:33:36.995219 19143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:36.967470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:45409 (local address 127.11.14.66:52085)
0807 18:33:36.968021 (+ 551us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:36.968077 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:36.969221 (+ 1144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:36.969812 (+ 591us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:36.969820 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:36.970422 (+ 602us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:33:36.971001 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:36.971010 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:36.972151 (+ 1141us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:36.972155 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:36.987201 (+ 15046us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:36.987213 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:36.987911 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:36.987918 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:36.988062 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:36.988535 (+ 473us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:33:36.988556 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:33:36.990211 (+ 1655us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:33:36.992811 (+ 2600us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:33:36.992814 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:33:36.992816 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:33:36.993066 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:33:36.993413 (+ 347us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:33:36.993416 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:33:36.993418 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:33:36.993540 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:33:36.994083 (+ 543us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:33:36.994086 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:33:36.994194 (+ 108us) client_negotiation.cc:769] Sending connection context
0807 18:33:36.994904 (+ 710us) client_negotiation.cc:240] Negotiation successful
0807 18:33:36.995066 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":326,"mutex_wait_us":225,"spinlock_wait_cycles":30336,"thread_start_us":129,"threads_started":1}
I20260807 18:33:36.996757 19138 heartbeater.cc:347] Connected to a master server at 127.11.14.124:36503
I20260807 18:33:36.996843 19138 heartbeater.cc:464] Registering TS with master...
I20260807 18:33:36.997105 19138 heartbeater.cc:511] Master 127.11.14.124:36503 requested a full tablet report, sending...
I20260807 18:33:36.997397 19139 heartbeater.cc:347] Connected to a master server at 127.11.14.126:45409
I20260807 18:33:36.997512 19139 heartbeater.cc:464] Registering TS with master...
I20260807 18:33:36.997716 19139 heartbeater.cc:511] Master 127.11.14.126:45409 requested a full tablet report, sending...
I20260807 18:33:36.997910 18796 ts_manager.cc:194] Registered new tserver with Master: f25e947f8c2c4fef895cfd78048211d7 (127.11.14.66:39193)
I20260807 18:33:36.998474 17963 ts_manager.cc:194] Re-registered known tserver with Master: f25e947f8c2c4fef895cfd78048211d7 (127.11.14.66:39193)
I20260807 18:33:36.999359 17963 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.11.14.66@KRBTEST.COM'} at 127.11.14.66:52085
I20260807 18:33:37.057538 19147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:37.053200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.124:36503 (local address 127.0.0.1:44600)
0807 18:33:37.053550 (+ 350us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:37.053564 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:37.053680 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:37.054017 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:37.054019 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:37.054036 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:33:37.054150 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:37.054155 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:37.055204 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:37.055207 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:37.056070 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:37.056089 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:37.057090 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:37.057093 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:37.057184 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:37.057206 (+ 22us) client_negotiation.cc:769] Sending connection context
0807 18:33:37.057262 (+ 56us) client_negotiation.cc:240] Negotiation successful
0807 18:33:37.057327 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":282,"thread_start_us":80,"threads_started":1}
I20260807 18:33:37.057564 18995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:37.053343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44600 (local address 127.11.14.124:36503)
0807 18:33:37.053504 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:37.053510 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:37.053643 (+ 133us) server_negotiation.cc:407] Connection header received
0807 18:33:37.053797 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:37.053801 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:37.053877 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:37.054008 (+ 131us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:33:37.054265 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:37.055063 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:37.056226 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:37.056956 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:37.057088 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:37.057277 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:37.057320 (+ 43us) server_negotiation.cc:299] Negotiation successful
0807 18:33:37.057418 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260807 18:33:37.058386 18807 raft_consensus.cc:1203] T 00000000000000000000000000000000 P ab43b0280a964c18a81ab0531a0a2a10 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260807 18:33:37.059001 18838 sys_catalog.cc:455] T 00000000000000000000000000000000 P ab43b0280a964c18a81ab0531a0a2a10 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9b176ea138c9402a95ef84541f6cc5e3. Latest consensus state: current_term: 1 leader_uuid: "9b176ea138c9402a95ef84541f6cc5e3" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "9b176ea138c9402a95ef84541f6cc5e3" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 45409 } } peers { permanent_uuid: "1abe9cee5c844198bbc7d32fdfae4374" member_type: VOTER last_known_addr { host: "127.11.14.125" port: 39483 } attrs { promote: false } } peers { permanent_uuid: "ab43b0280a964c18a81ab0531a0a2a10" member_type: NON_VOTER last_known_addr { host: "127.11.14.124" port: 36503 } attrs { promote: true } } }
I20260807 18:33:37.059163 18838 sys_catalog.cc:458] T 00000000000000000000000000000000 P ab43b0280a964c18a81ab0531a0a2a10 [sys.catalog]: This master's current role is: LEARNER
W20260807 18:33:37.121264 19146 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:33:37.121521 19146 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:33:37.121567 19146 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:33:37.125734 19146 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:33:37.125813 19146 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:33:37.125902 19146 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.14.67
I20260807 18:33:37.131810 19146 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.14.84:41437
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/krb5kdc/oryx_127.11.14.67.keytab
--principal=oryx/127.11.14.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.14.67:42159
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.14.67
--webserver_port=40269
--tserver_master_addrs=127.11.14.126:45409,127.11.14.125:39483,127.11.14.124:36503
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19146
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.14.67
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:33:37.132827 19146 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:33:37.133879 19146 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:33:37.144011 19155 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:33:37.143980 19152 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:33:37.144068 19153 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:33:37.144210 19146 server_base.cc:1034] running on GCE node
I20260807 18:33:37.144716 19146 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:33:37.145364 19146 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:33:37.146590 19146 hybrid_clock.cc:648] HybridClock initialized: now 1786127617146570 us; error 86 us; skew 500 ppm
Aug 07 18:33:37 dist-test-slave-6k22 krb5kdc[17915](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127617, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.14.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:33:37.149713 19146 init.cc:377] Logged in from keytab as oryx/127.11.14.67@KRBTEST.COM (short username oryx)
I20260807 18:33:37.150626 19146 webserver.cc:460] Webserver started at http://127.11.14.67:40269/ using document root <none> and password file <none>
I20260807 18:33:37.151218 19146 fs_manager.cc:359] Metadata directory not provided
I20260807 18:33:37.151345 19146 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:33:37.155407 19146 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:33:37.157648 19163 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:33:37.159157 19146 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 18:33:37.159291 19146 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/minicluster-data/ts-2/data,/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/minicluster-data/ts-2/wal
uuid: "6a3774b03adf44d3b1475982c04da6aa"
format_stamp: "Formatted at 2026-08-07 18:33:35 on dist-test-slave-6k22"
server_key: "695e881c8d55f5d75f1119a7b5ac9b7d"
server_key_iv: "59e13a5b0d5cda5d1fe52204662cac99"
server_key_version: "encryptionkey@0"
I20260807 18:33:37.159885 19146 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:33:37.197127 19146 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:33:37.202317 19146 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:33:37.202484 19146 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:33:37.203094 19146 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:33:37.204043 19146 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:33:37.204093 19146 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:33:37.204133 19146 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:33:37.204149 19146 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:33:37.214350 19146 rpc_server.cc:307] RPC server started. Bound to: 127.11.14.67:42159
I20260807 18:33:37.214383 19285 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.14.67:42159 every 8 connection(s)
I20260807 18:33:37.215373 19146 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/minicluster-data/ts-2/data/info.pb
I20260807 18:33:37.222618 11321 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkz56XN/build/debug/bin/kudu as pid 19146
Aug 07 18:33:37 dist-test-slave-6k22 krb5kdc[17915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127617, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.14.67@KRBTEST.COM for oryx/127.11.14.126@KRBTEST.COM
Aug 07 18:33:37 dist-test-slave-6k22 krb5kdc[17915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127617, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.14.67@KRBTEST.COM for oryx/127.11.14.124@KRBTEST.COM
Aug 07 18:33:37 dist-test-slave-6k22 krb5kdc[17915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127617, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.14.67@KRBTEST.COM for oryx/127.11.14.125@KRBTEST.COM
I20260807 18:33:37.233610 18995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:37.217964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.67:35275 (local address 127.11.14.124:36503)
0807 18:33:37.218125 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:37.218140 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:37.219347 (+ 1207us) server_negotiation.cc:407] Connection header received
0807 18:33:37.219810 (+ 463us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:37.219814 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:37.219875 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:37.219959 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:33:37.221344 (+ 1385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:37.222184 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:37.223394 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:37.223740 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:37.223852 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:37.228259 (+ 4407us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:33:37.228263 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:33:37.228265 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:33:37.228287 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:33:37.231153 (+ 2866us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:37.231735 (+ 582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:37.231739 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:37.231741 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:37.231828 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:37.232205 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:37.232211 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:37.232215 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:37.232450 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:33:37.232550 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:37.233289 (+ 739us) server_negotiation.cc:299] Negotiation successful
0807 18:33:37.233419 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260807 18:33:37.234795 19293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:37.217816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.124:36503 (local address 127.11.14.67:35275)
0807 18:33:37.219229 (+ 1413us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:37.219245 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:37.219594 (+ 349us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:37.220138 (+ 544us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:37.220146 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:37.220602 (+ 456us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:33:37.221183 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:37.221198 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:37.222360 (+ 1162us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:37.222363 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:37.223255 (+ 892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:37.223265 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:37.224041 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:37.224045 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:37.224352 (+ 307us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:37.225154 (+ 802us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:33:37.225184 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:33:37.228076 (+ 2892us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:33:37.231273 (+ 3197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:33:37.231276 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:33:37.231278 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:33:37.231615 (+ 337us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:33:37.231939 (+ 324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:33:37.231942 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:33:37.231944 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:33:37.232069 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:33:37.232635 (+ 566us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:33:37.232642 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:33:37.233007 (+ 365us) client_negotiation.cc:769] Sending connection context
0807 18:33:37.234025 (+ 1018us) client_negotiation.cc:240] Negotiation successful
0807 18:33:37.234187 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1299,"mutex_wait_us":92,"spinlock_wait_cycles":51328,"thread_start_us":212,"threads_started":1}
I20260807 18:33:37.234925 18730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:37.219109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.67:42009 (local address 127.11.14.125:39483)
0807 18:33:37.219231 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:37.219237 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:37.219405 (+ 168us) server_negotiation.cc:407] Connection header received
0807 18:33:37.219843 (+ 438us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:37.219846 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:37.219901 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:37.219955 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:33:37.221397 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:37.222185 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:37.226856 (+ 4671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:37.227219 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:37.227296 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:37.230442 (+ 3146us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:33:37.230446 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:33:37.230449 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:33:37.230480 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:33:37.233058 (+ 2578us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:37.233619 (+ 561us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:37.233623 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:37.233625 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:37.233715 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:37.234011 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:37.234014 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:37.234016 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:37.234215 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:33:37.234296 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:37.234641 (+ 345us) server_negotiation.cc:299] Negotiation successful
0807 18:33:37.234755 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260807 18:33:37.235092 19291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:37.217816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:45409 (local address 127.11.14.67:37493)
0807 18:33:37.218422 (+ 606us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:37.218475 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:37.219615 (+ 1140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:37.220139 (+ 524us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:37.220147 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:37.220659 (+ 512us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:33:37.221206 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:37.221214 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:37.222072 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:37.222075 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:37.223326 (+ 1251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:37.223338 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:37.224040 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:37.224044 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:37.224348 (+ 304us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:37.225203 (+ 855us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:33:37.225222 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:33:37.228055 (+ 2833us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:33:37.231040 (+ 2985us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:33:37.231048 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:33:37.231068 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:33:37.231460 (+ 392us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:33:37.231862 (+ 402us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:33:37.231867 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:33:37.231869 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:33:37.232072 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:33:37.232639 (+ 567us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:33:37.232645 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:33:37.233005 (+ 360us) client_negotiation.cc:769] Sending connection context
0807 18:33:37.233222 (+ 217us) client_negotiation.cc:240] Negotiation successful
0807 18:33:37.233619 (+ 397us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":372,"mutex_wait_us":195,"spinlock_wait_cycles":31360,"thread_start_us":666,"threads_started":1}
I20260807 18:33:37.235471 19292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:37.218977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.125:39483 (local address 127.11.14.67:42009)
0807 18:33:37.219313 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:37.219325 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:37.219624 (+ 299us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:37.220148 (+ 524us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:37.220151 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:37.220665 (+ 514us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:33:37.221205 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:37.221214 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:37.225233 (+ 4019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:37.225236 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:37.226703 (+ 1467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:37.226716 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:37.227469 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:37.227472 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:37.227535 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:37.228885 (+ 1350us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:33:37.228903 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:33:37.230269 (+ 1366us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:33:37.233185 (+ 2916us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:33:37.233189 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:33:37.233194 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:33:37.233490 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:33:37.233821 (+ 331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:33:37.233824 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:33:37.233825 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:33:37.233908 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:33:37.234367 (+ 459us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:33:37.234370 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:33:37.234455 (+ 85us) client_negotiation.cc:769] Sending connection context
0807 18:33:37.235138 (+ 683us) client_negotiation.cc:240] Negotiation successful
0807 18:33:37.235277 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":277,"mutex_wait_us":111,"spinlock_wait_cycles":1792,"thread_start_us":79,"threads_started":1}
I20260807 18:33:37.235486 18004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:37.217382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.67:37493 (local address 127.11.14.126:45409)
0807 18:33:37.217534 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:37.217538 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:37.218591 (+ 1053us) server_negotiation.cc:407] Connection header received
0807 18:33:37.219842 (+ 1251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:37.219846 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:37.219901 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:37.219955 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:33:37.221389 (+ 1434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:37.221924 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:37.223501 (+ 1577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:37.223887 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:37.223996 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:37.228259 (+ 4263us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:33:37.228263 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:33:37.228266 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:33:37.228298 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:33:37.230922 (+ 2624us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:37.231618 (+ 696us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:37.231622 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:37.231624 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:37.231722 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:37.232196 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:37.232201 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:37.232203 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:37.232478 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:33:37.232584 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:37.235225 (+ 2641us) server_negotiation.cc:299] Negotiation successful
0807 18:33:37.235342 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260807 18:33:37.236990 19288 heartbeater.cc:347] Connected to a master server at 127.11.14.126:45409
I20260807 18:33:37.237028 19289 heartbeater.cc:347] Connected to a master server at 127.11.14.125:39483
I20260807 18:33:37.237164 19286 heartbeater.cc:347] Connected to a master server at 127.11.14.124:36503
I20260807 18:33:37.237267 19289 heartbeater.cc:464] Registering TS with master...
I20260807 18:33:37.237947 19289 heartbeater.cc:511] Master 127.11.14.125:39483 requested a full tablet report, sending...
I20260807 18:33:37.237267 19286 heartbeater.cc:464] Registering TS with master...
I20260807 18:33:37.239037 19286 heartbeater.cc:511] Master 127.11.14.124:36503 requested a full tablet report, sending...
I20260807 18:33:37.240091 18796 ts_manager.cc:194] Registered new tserver with Master: 6a3774b03adf44d3b1475982c04da6aa (127.11.14.67:42159)
I20260807 18:33:37.240092 18600 ts_manager.cc:194] Registered new tserver with Master: 6a3774b03adf44d3b1475982c04da6aa (127.11.14.67:42159)
I20260807 18:33:37.240420 19301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:37.224914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54270 (local address 127.11.14.125:39483)
0807 18:33:37.225748 (+ 834us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:37.225753 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:37.225768 (+ 15us) server_negotiation.cc:407] Connection header received
0807 18:33:37.225811 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:37.225813 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:37.225861 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:37.225962 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:33:37.227087 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:37.228402 (+ 1315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:37.230376 (+ 1974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:37.230771 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:37.231273 (+ 502us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:37.233415 (+ 2142us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:33:37.233418 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:33:37.233421 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:33:37.233444 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:33:37.238038 (+ 4594us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:37.238602 (+ 564us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:37.238605 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:37.238607 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:37.238692 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:37.239041 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:37.239046 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:37.239050 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:37.239260 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:33:37.239452 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:37.240009 (+ 557us) server_negotiation.cc:299] Negotiation successful
0807 18:33:37.240221 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":751,"thread_start_us":78,"threads_started":1}
I20260807 18:33:37.241937 19303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:37.225900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44610 (local address 127.11.14.124:36503)
0807 18:33:37.227224 (+ 1324us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:37.227228 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:37.228017 (+ 789us) server_negotiation.cc:407] Connection header received
0807 18:33:37.228063 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:37.228066 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:37.228116 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:37.228249 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:33:37.231423 (+ 3174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:37.233289 (+ 1866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:37.234745 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:37.235095 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:37.235179 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:37.237342 (+ 2163us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:33:37.237348 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:33:37.237352 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:33:37.237378 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:33:37.239202 (+ 1824us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:37.240386 (+ 1184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:37.240389 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:37.240391 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:37.240485 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:37.241026 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:37.241030 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:37.241032 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:37.241237 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:33:37.241382 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:37.241645 (+ 263us) server_negotiation.cc:299] Negotiation successful
0807 18:33:37.241771 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1246,"thread_start_us":106,"threads_started":1}
I20260807 18:33:37.237268 19288 heartbeater.cc:464] Registering TS with master...
I20260807 18:33:37.244120 19288 heartbeater.cc:511] Master 127.11.14.126:45409 requested a full tablet report, sending...
I20260807 18:33:37.244801 17963 ts_manager.cc:194] Re-registered known tserver with Master: 6a3774b03adf44d3b1475982c04da6aa (127.11.14.67:42159)
I20260807 18:33:37.245505 17963 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.11.14.67@KRBTEST.COM'} at 127.11.14.67:37493
I20260807 18:33:37.248588 18004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:37.236308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32994 (local address 127.11.14.126:45409)
0807 18:33:37.236675 (+ 367us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:37.236679 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:37.237845 (+ 1166us) server_negotiation.cc:407] Connection header received
0807 18:33:37.238044 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:37.238047 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:37.238096 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:37.238152 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:33:37.239138 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:37.239898 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:37.242731 (+ 2833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:37.243068 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:37.243169 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:37.244750 (+ 1581us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:33:37.244754 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:33:37.244756 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:33:37.244782 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:33:37.246854 (+ 2072us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:37.247308 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:37.247311 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:37.247312 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:37.247387 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:37.247686 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:37.247688 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:37.247689 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:37.247837 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:33:37.247933 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:37.248175 (+ 242us) server_negotiation.cc:299] Negotiation successful
0807 18:33:37.248308 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":297}
I20260807 18:33:37.251322 17963 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:32994:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 18:33:37.254585 17963 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 18:33:37.257282 18753 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b176ea138c9402a95ef84541f6cc5e3 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9b176ea138c9402a95ef84541f6cc5e3" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "9b176ea138c9402a95ef84541f6cc5e3" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 45409 } } peers { permanent_uuid: "1abe9cee5c844198bbc7d32fdfae4374" member_type: VOTER last_known_addr { host: "127.11.14.125" port: 39483 } attrs { promote: false } } peers { permanent_uuid: "ab43b0280a964c18a81ab0531a0a2a10" member_type: NON_VOTER last_known_addr { host: "127.11.14.124" port: 36503 } attrs { promote: true } } }
I20260807 18:33:37.257431 18753 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b176ea138c9402a95ef84541f6cc5e3 [sys.catalog]: This master's current role is: LEADER
I20260807 18:33:37.261133 18753 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 9b176ea138c9402a95ef84541f6cc5e3: attempting to promote NON_VOTER ab43b0280a964c18a81ab0531a0a2a10 to VOTER
I20260807 18:33:37.261467 18753 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9b176ea138c9402a95ef84541f6cc5e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "9b176ea138c9402a95ef84541f6cc5e3" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 45409 } } peers { permanent_uuid: "1abe9cee5c844198bbc7d32fdfae4374" member_type: VOTER last_known_addr { host: "127.11.14.125" port: 39483 } attrs { promote: false } } peers { permanent_uuid: "ab43b0280a964c18a81ab0531a0a2a10" member_type: VOTER last_known_addr { host: "127.11.14.124" port: 36503 } attrs { promote: false } }
I20260807 18:33:37.262887 18807 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ab43b0280a964c18a81ab0531a0a2a10 [term 1 LEARNER]: Refusing update from remote peer 9b176ea138c9402a95ef84541f6cc5e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260807 18:33:37.262993 18611 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1abe9cee5c844198bbc7d32fdfae4374 [term 1 FOLLOWER]: Refusing update from remote peer 9b176ea138c9402a95ef84541f6cc5e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260807 18:33:37.263470 19309 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 9b176ea138c9402a95ef84541f6cc5e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ab43b0280a964c18a81ab0531a0a2a10" member_type: VOTER last_known_addr { host: "127.11.14.124" port: 36503 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260807 18:33:37.263616 19307 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 9b176ea138c9402a95ef84541f6cc5e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1abe9cee5c844198bbc7d32fdfae4374" member_type: VOTER last_known_addr { host: "127.11.14.125" port: 39483 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260807 18:33:37.265261 19309 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 9b176ea138c9402a95ef84541f6cc5e3 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 7 to 9, ab43b0280a964c18a81ab0531a0a2a10 (127.11.14.124) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "9b176ea138c9402a95ef84541f6cc5e3" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 45409 } } peers { permanent_uuid: "1abe9cee5c844198bbc7d32fdfae4374" member_type: VOTER last_known_addr { host: "127.11.14.125" port: 39483 } attrs { promote: false } } peers { permanent_uuid: "ab43b0280a964c18a81ab0531a0a2a10" member_type: VOTER last_known_addr { host: "127.11.14.124" port: 36503 } attrs { promote: false } } }
I20260807 18:33:37.266006 18807 raft_consensus.cc:2932] T 00000000000000000000000000000000 P ab43b0280a964c18a81ab0531a0a2a10 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, ab43b0280a964c18a81ab0531a0a2a10 (127.11.14.124) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "9b176ea138c9402a95ef84541f6cc5e3" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 45409 } } peers { permanent_uuid: "1abe9cee5c844198bbc7d32fdfae4374" member_type: VOTER last_known_addr { host: "127.11.14.125" port: 39483 } attrs { promote: false } } peers { permanent_uuid: "ab43b0280a964c18a81ab0531a0a2a10" member_type: VOTER last_known_addr { host: "127.11.14.124" port: 36503 } attrs { promote: false } } }
I20260807 18:33:37.266727 19309 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b176ea138c9402a95ef84541f6cc5e3 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "9b176ea138c9402a95ef84541f6cc5e3" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "9b176ea138c9402a95ef84541f6cc5e3" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 45409 } } peers { permanent_uuid: "1abe9cee5c844198bbc7d32fdfae4374" member_type: VOTER last_known_addr { host: "127.11.14.125" port: 39483 } attrs { promote: false } } peers { permanent_uuid: "ab43b0280a964c18a81ab0531a0a2a10" member_type: VOTER last_known_addr { host: "127.11.14.124" port: 36503 } attrs { promote: false } } }
I20260807 18:33:37.266840 19309 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b176ea138c9402a95ef84541f6cc5e3 [sys.catalog]: This master's current role is: LEADER
I20260807 18:33:37.266906 19308 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b176ea138c9402a95ef84541f6cc5e3 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9b176ea138c9402a95ef84541f6cc5e3" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "9b176ea138c9402a95ef84541f6cc5e3" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 45409 } } peers { permanent_uuid: "1abe9cee5c844198bbc7d32fdfae4374" member_type: VOTER last_known_addr { host: "127.11.14.125" port: 39483 } attrs { promote: false } } peers { permanent_uuid: "ab43b0280a964c18a81ab0531a0a2a10" member_type: VOTER last_known_addr { host: "127.11.14.124" port: 36503 } attrs { promote: false } } }
I20260807 18:33:37.266994 19308 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b176ea138c9402a95ef84541f6cc5e3 [sys.catalog]: This master's current role is: LEADER
I20260807 18:33:37.267189 19309 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b176ea138c9402a95ef84541f6cc5e3 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9b176ea138c9402a95ef84541f6cc5e3" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "9b176ea138c9402a95ef84541f6cc5e3" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 45409 } } peers { permanent_uuid: "1abe9cee5c844198bbc7d32fdfae4374" member_type: VOTER last_known_addr { host: "127.11.14.125" port: 39483 } attrs { promote: false } } peers { permanent_uuid: "ab43b0280a964c18a81ab0531a0a2a10" member_type: VOTER last_known_addr { host: "127.11.14.124" port: 36503 } attrs { promote: false } } }
I20260807 18:33:37.267280 19309 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b176ea138c9402a95ef84541f6cc5e3 [sys.catalog]: This master's current role is: LEADER
I20260807 18:33:37.267793 18611 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 1abe9cee5c844198bbc7d32fdfae4374 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, ab43b0280a964c18a81ab0531a0a2a10 (127.11.14.124) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "9b176ea138c9402a95ef84541f6cc5e3" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 45409 } } peers { permanent_uuid: "1abe9cee5c844198bbc7d32fdfae4374" member_type: VOTER last_known_addr { host: "127.11.14.125" port: 39483 } attrs { promote: false } } peers { permanent_uuid: "ab43b0280a964c18a81ab0531a0a2a10" member_type: VOTER last_known_addr { host: "127.11.14.124" port: 36503 } attrs { promote: false } } }
I20260807 18:33:37.268359 18838 sys_catalog.cc:455] T 00000000000000000000000000000000 P ab43b0280a964c18a81ab0531a0a2a10 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9b176ea138c9402a95ef84541f6cc5e3" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "9b176ea138c9402a95ef84541f6cc5e3" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 45409 } } peers { permanent_uuid: "1abe9cee5c844198bbc7d32fdfae4374" member_type: VOTER last_known_addr { host: "127.11.14.125" port: 39483 } attrs { promote: false } } peers { permanent_uuid: "ab43b0280a964c18a81ab0531a0a2a10" member_type: VOTER last_known_addr { host: "127.11.14.124" port: 36503 } attrs { promote: false } } }
I20260807 18:33:37.268499 18838 sys_catalog.cc:458] T 00000000000000000000000000000000 P ab43b0280a964c18a81ab0531a0a2a10 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 18:33:37.269708 19317 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1abe9cee5c844198bbc7d32fdfae4374 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9b176ea138c9402a95ef84541f6cc5e3" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "9b176ea138c9402a95ef84541f6cc5e3" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 45409 } } peers { permanent_uuid: "1abe9cee5c844198bbc7d32fdfae4374" member_type: VOTER last_known_addr { host: "127.11.14.125" port: 39483 } attrs { promote: false } } peers { permanent_uuid: "ab43b0280a964c18a81ab0531a0a2a10" member_type: VOTER last_known_addr { host: "127.11.14.124" port: 36503 } attrs { promote: false } } }
I20260807 18:33:37.269917 19317 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1abe9cee5c844198bbc7d32fdfae4374 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 18:33:37.278169 19147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:37.272617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.66:39193 (local address 127.0.0.1:36710)
0807 18:33:37.272783 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:37.272797 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:37.272865 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:37.273825 (+ 960us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:37.273830 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:37.273845 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:33:37.273953 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:37.273959 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:37.275466 (+ 1507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:37.275470 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:37.276670 (+ 1200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:37.276682 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:37.277745 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:37.277749 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:37.277837 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:37.277862 (+ 25us) client_negotiation.cc:769] Sending connection context
0807 18:33:37.277933 (+ 71us) client_negotiation.cc:240] Negotiation successful
0807 18:33:37.278016 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":89}
I20260807 18:33:37.278237 19318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:37.272814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36710 (local address 127.11.14.66:39193)
0807 18:33:37.273176 (+ 362us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:37.273186 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:37.273208 (+ 22us) server_negotiation.cc:407] Connection header received
0807 18:33:37.273311 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:37.273316 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:37.273593 (+ 277us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:37.274131 (+ 538us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:33:37.274294 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:37.275348 (+ 1054us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:37.276809 (+ 1461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:37.277610 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:37.277750 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:37.277926 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:37.277983 (+ 57us) server_negotiation.cc:299] Negotiation successful
0807 18:33:37.278083 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":263,"thread_start_us":150,"threads_started":1}
I20260807 18:33:37.279462 19321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:37.273763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.65:33879 (local address 127.0.0.1:60094)
0807 18:33:37.273981 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:37.273995 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:37.274071 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:37.275063 (+ 992us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:37.275066 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:37.275085 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:33:37.275182 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:37.275187 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:37.276632 (+ 1445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:37.276635 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:37.277769 (+ 1134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:37.277780 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:37.279067 (+ 1287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:37.279071 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:37.279174 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:37.279196 (+ 22us) client_negotiation.cc:769] Sending connection context
0807 18:33:37.279266 (+ 70us) client_negotiation.cc:240] Negotiation successful
0807 18:33:37.279329 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":96,"threads_started":1}
I20260807 18:33:37.280048 19322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:37.274385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60094 (local address 127.11.14.65:33879)
0807 18:33:37.274675 (+ 290us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:37.274680 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:37.274701 (+ 21us) server_negotiation.cc:407] Connection header received
0807 18:33:37.274771 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:37.274775 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:37.274925 (+ 150us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:37.275051 (+ 126us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:33:37.275280 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:37.276491 (+ 1211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:37.277930 (+ 1439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:37.278754 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:37.278866 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:37.279001 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:37.279784 (+ 783us) server_negotiation.cc:299] Negotiation successful
0807 18:33:37.279901 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":180,"thread_start_us":93,"threads_started":1}
I20260807 18:33:37.280191 19319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:37.273306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.67:42159 (local address 127.0.0.1:40382)
0807 18:33:37.273611 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:37.273624 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:37.273712 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:37.276048 (+ 2336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:37.276051 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:37.276069 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:33:37.276159 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:37.276168 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:37.277446 (+ 1278us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:37.277450 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:37.278691 (+ 1241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:37.278704 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:37.279833 (+ 1129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:37.279837 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:37.279921 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:37.279943 (+ 22us) client_negotiation.cc:769] Sending connection context
0807 18:33:37.279993 (+ 50us) client_negotiation.cc:240] Negotiation successful
0807 18:33:37.280053 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":151,"threads_started":1}
I20260807 18:33:37.281481 19068 tablet_service.cc:1467] Processing CreateTablet for tablet c121358a3dd441ff9162618b95625837 (DEFAULT_TABLE table=test-table [id=9d3f6f025d5d45c4a9e9fa9f392ecd6e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:33:37.282739 19068 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c121358a3dd441ff9162618b95625837. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:33:37.282428 18919 tablet_service.cc:1467] Processing CreateTablet for tablet c121358a3dd441ff9162618b95625837 (DEFAULT_TABLE table=test-table [id=9d3f6f025d5d45c4a9e9fa9f392ecd6e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:33:37.283258 18919 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c121358a3dd441ff9162618b95625837. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:33:37.283756 19320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:37.273520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40382 (local address 127.11.14.67:42159)
0807 18:33:37.275655 (+ 2135us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:37.275673 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:37.275693 (+ 20us) server_negotiation.cc:407] Connection header received
0807 18:33:37.275757 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:37.275760 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:37.275908 (+ 148us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:37.276062 (+ 154us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:33:37.276257 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:37.277299 (+ 1042us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:37.278853 (+ 1554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:37.279702 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:37.280452 (+ 750us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:37.280606 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:37.280655 (+ 49us) server_negotiation.cc:299] Negotiation successful
0807 18:33:37.280737 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1978,"thread_start_us":163,"threads_started":1}
I20260807 18:33:37.286406 19214 tablet_service.cc:1467] Processing CreateTablet for tablet c121358a3dd441ff9162618b95625837 (DEFAULT_TABLE table=test-table [id=9d3f6f025d5d45c4a9e9fa9f392ecd6e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:33:37.287614 19214 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c121358a3dd441ff9162618b95625837. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:33:37.290017 19323 tablet_bootstrap.cc:492] T c121358a3dd441ff9162618b95625837 P a290fe9b958d48f0a540e728e58c4bc7: Bootstrap starting.
I20260807 18:33:37.291250 19324 tablet_bootstrap.cc:492] T c121358a3dd441ff9162618b95625837 P f25e947f8c2c4fef895cfd78048211d7: Bootstrap starting.
I20260807 18:33:37.292598 19323 tablet_bootstrap.cc:654] T c121358a3dd441ff9162618b95625837 P a290fe9b958d48f0a540e728e58c4bc7: Neither blocks nor log segments found. Creating new log.
I20260807 18:33:37.293444 19324 tablet_bootstrap.cc:654] T c121358a3dd441ff9162618b95625837 P f25e947f8c2c4fef895cfd78048211d7: Neither blocks nor log segments found. Creating new log.
I20260807 18:33:37.293646 19323 log.cc:824] T c121358a3dd441ff9162618b95625837 P a290fe9b958d48f0a540e728e58c4bc7: Log is configured to *not* fsync() on all Append() calls
I20260807 18:33:37.294260 19324 log.cc:824] T c121358a3dd441ff9162618b95625837 P f25e947f8c2c4fef895cfd78048211d7: Log is configured to *not* fsync() on all Append() calls
I20260807 18:33:37.295826 19323 tablet_bootstrap.cc:492] T c121358a3dd441ff9162618b95625837 P a290fe9b958d48f0a540e728e58c4bc7: No bootstrap required, opened a new log
I20260807 18:33:37.296043 19327 tablet_bootstrap.cc:492] T c121358a3dd441ff9162618b95625837 P 6a3774b03adf44d3b1475982c04da6aa: Bootstrap starting.
I20260807 18:33:37.296106 19323 ts_tablet_manager.cc:1397] T c121358a3dd441ff9162618b95625837 P a290fe9b958d48f0a540e728e58c4bc7: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.001s
I20260807 18:33:37.296423 19324 tablet_bootstrap.cc:492] T c121358a3dd441ff9162618b95625837 P f25e947f8c2c4fef895cfd78048211d7: No bootstrap required, opened a new log
I20260807 18:33:37.296615 19324 ts_tablet_manager.cc:1397] T c121358a3dd441ff9162618b95625837 P f25e947f8c2c4fef895cfd78048211d7: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260807 18:33:37.297858 19327 tablet_bootstrap.cc:654] T c121358a3dd441ff9162618b95625837 P 6a3774b03adf44d3b1475982c04da6aa: Neither blocks nor log segments found. Creating new log.
I20260807 18:33:37.298571 19327 log.cc:824] T c121358a3dd441ff9162618b95625837 P 6a3774b03adf44d3b1475982c04da6aa: Log is configured to *not* fsync() on all Append() calls
I20260807 18:33:37.300609 19327 tablet_bootstrap.cc:492] T c121358a3dd441ff9162618b95625837 P 6a3774b03adf44d3b1475982c04da6aa: No bootstrap required, opened a new log
I20260807 18:33:37.300480 19323 raft_consensus.cc:348] T c121358a3dd441ff9162618b95625837 P a290fe9b958d48f0a540e728e58c4bc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f25e947f8c2c4fef895cfd78048211d7" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 39193 } } peers { permanent_uuid: "6a3774b03adf44d3b1475982c04da6aa" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 42159 } } peers { permanent_uuid: "a290fe9b958d48f0a540e728e58c4bc7" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 33879 } }
I20260807 18:33:37.300755 19323 raft_consensus.cc:374] T c121358a3dd441ff9162618b95625837 P a290fe9b958d48f0a540e728e58c4bc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:33:37.300815 19327 ts_tablet_manager.cc:1397] T c121358a3dd441ff9162618b95625837 P 6a3774b03adf44d3b1475982c04da6aa: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260807 18:33:37.300823 19323 raft_consensus.cc:729] T c121358a3dd441ff9162618b95625837 P a290fe9b958d48f0a540e728e58c4bc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a290fe9b958d48f0a540e728e58c4bc7, State: Initialized, Role: FOLLOWER
I20260807 18:33:37.301110 19324 raft_consensus.cc:348] T c121358a3dd441ff9162618b95625837 P f25e947f8c2c4fef895cfd78048211d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f25e947f8c2c4fef895cfd78048211d7" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 39193 } } peers { permanent_uuid: "6a3774b03adf44d3b1475982c04da6aa" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 42159 } } peers { permanent_uuid: "a290fe9b958d48f0a540e728e58c4bc7" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 33879 } }
I20260807 18:33:37.301359 19324 raft_consensus.cc:374] T c121358a3dd441ff9162618b95625837 P f25e947f8c2c4fef895cfd78048211d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:33:37.301288 19323 consensus_queue.cc:260] T c121358a3dd441ff9162618b95625837 P a290fe9b958d48f0a540e728e58c4bc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f25e947f8c2c4fef895cfd78048211d7" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 39193 } } peers { permanent_uuid: "6a3774b03adf44d3b1475982c04da6aa" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 42159 } } peers { permanent_uuid: "a290fe9b958d48f0a540e728e58c4bc7" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 33879 } }
I20260807 18:33:37.301415 19324 raft_consensus.cc:729] T c121358a3dd441ff9162618b95625837 P f25e947f8c2c4fef895cfd78048211d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f25e947f8c2c4fef895cfd78048211d7, State: Initialized, Role: FOLLOWER
I20260807 18:33:37.301887 19324 consensus_queue.cc:260] T c121358a3dd441ff9162618b95625837 P f25e947f8c2c4fef895cfd78048211d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f25e947f8c2c4fef895cfd78048211d7" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 39193 } } peers { permanent_uuid: "6a3774b03adf44d3b1475982c04da6aa" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 42159 } } peers { permanent_uuid: "a290fe9b958d48f0a540e728e58c4bc7" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 33879 } }
I20260807 18:33:37.302305 19323 ts_tablet_manager.cc:1428] T c121358a3dd441ff9162618b95625837 P a290fe9b958d48f0a540e728e58c4bc7: Time spent starting tablet: real 0.006s user 0.003s sys 0.002s
I20260807 18:33:37.302574 18992 heartbeater.cc:503] Master 127.11.14.126:45409 was elected leader, sending a full tablet report...
I20260807 18:33:37.302837 19324 ts_tablet_manager.cc:1428] T c121358a3dd441ff9162618b95625837 P f25e947f8c2c4fef895cfd78048211d7: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260807 18:33:37.304606 19139 heartbeater.cc:503] Master 127.11.14.126:45409 was elected leader, sending a full tablet report...
I20260807 18:33:37.305594 19327 raft_consensus.cc:348] T c121358a3dd441ff9162618b95625837 P 6a3774b03adf44d3b1475982c04da6aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f25e947f8c2c4fef895cfd78048211d7" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 39193 } } peers { permanent_uuid: "6a3774b03adf44d3b1475982c04da6aa" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 42159 } } peers { permanent_uuid: "a290fe9b958d48f0a540e728e58c4bc7" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 33879 } }
I20260807 18:33:37.305851 19327 raft_consensus.cc:374] T c121358a3dd441ff9162618b95625837 P 6a3774b03adf44d3b1475982c04da6aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:33:37.305922 19327 raft_consensus.cc:729] T c121358a3dd441ff9162618b95625837 P 6a3774b03adf44d3b1475982c04da6aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a3774b03adf44d3b1475982c04da6aa, State: Initialized, Role: FOLLOWER
I20260807 18:33:37.306386 19327 consensus_queue.cc:260] T c121358a3dd441ff9162618b95625837 P 6a3774b03adf44d3b1475982c04da6aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f25e947f8c2c4fef895cfd78048211d7" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 39193 } } peers { permanent_uuid: "6a3774b03adf44d3b1475982c04da6aa" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 42159 } } peers { permanent_uuid: "a290fe9b958d48f0a540e728e58c4bc7" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 33879 } }
I20260807 18:33:37.307384 19327 ts_tablet_manager.cc:1428] T c121358a3dd441ff9162618b95625837 P 6a3774b03adf44d3b1475982c04da6aa: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260807 18:33:37.307770 19288 heartbeater.cc:503] Master 127.11.14.126:45409 was elected leader, sending a full tablet report...
I20260807 18:33:37.392884 19329 raft_consensus.cc:482] T c121358a3dd441ff9162618b95625837 P a290fe9b958d48f0a540e728e58c4bc7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:33:37.393124 19329 raft_consensus.cc:504] T c121358a3dd441ff9162618b95625837 P a290fe9b958d48f0a540e728e58c4bc7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f25e947f8c2c4fef895cfd78048211d7" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 39193 } } peers { permanent_uuid: "6a3774b03adf44d3b1475982c04da6aa" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 42159 } } peers { permanent_uuid: "a290fe9b958d48f0a540e728e58c4bc7" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 33879 } }
I20260807 18:33:37.394436 19329 leader_election.cc:288] T c121358a3dd441ff9162618b95625837 P a290fe9b958d48f0a540e728e58c4bc7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f25e947f8c2c4fef895cfd78048211d7 (127.11.14.66:39193), 6a3774b03adf44d3b1475982c04da6aa (127.11.14.67:42159)
I20260807 18:33:37.398607 19320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:37.394870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.65:60289 (local address 127.11.14.67:42159)
0807 18:33:37.394998 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:37.395003 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:37.395287 (+ 284us) server_negotiation.cc:407] Connection header received
0807 18:33:37.395426 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:37.395429 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:37.395475 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:37.395585 (+ 110us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:33:37.395906 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:37.396419 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:37.397592 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:37.397989 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:37.398084 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:37.398166 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:37.398400 (+ 234us) server_negotiation.cc:299] Negotiation successful
0807 18:33:37.398472 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260807 18:33:37.398638 19332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:37.394596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.66:39193 (local address 127.11.14.65:43901)
0807 18:33:37.395058 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:37.395071 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:37.395171 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:37.395425 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:37.395429 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:37.395447 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:33:37.395623 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:37.395631 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:37.396524 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:37.396527 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:37.397461 (+ 934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:37.397471 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:37.398173 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:37.398180 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:37.398271 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:37.398288 (+ 17us) client_negotiation.cc:769] Sending connection context
0807 18:33:37.398399 (+ 111us) client_negotiation.cc:240] Negotiation successful
0807 18:33:37.398469 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":386,"mutex_wait_us":72,"thread_start_us":178,"threads_started":1}
I20260807 18:33:37.398660 19333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:37.394733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.67:42159 (local address 127.11.14.65:60289)
0807 18:33:37.395165 (+ 432us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:37.395182 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:37.395292 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:37.395621 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:37.395624 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:37.395641 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:33:37.395780 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:37.395787 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:37.396569 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:37.396572 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:37.397457 (+ 885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:37.397465 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:37.398127 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:37.398131 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:37.398230 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:37.398257 (+ 27us) client_negotiation.cc:769] Sending connection context
0807 18:33:37.398334 (+ 77us) client_negotiation.cc:240] Negotiation successful
0807 18:33:37.398433 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":341,"mutex_wait_us":4,"thread_start_us":109,"threads_started":1}
I20260807 18:33:37.398648 19318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:37.394741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.65:43901 (local address 127.11.14.66:39193)
0807 18:33:37.394888 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:37.394893 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:37.395179 (+ 286us) server_negotiation.cc:407] Connection header received
0807 18:33:37.395256 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:37.395259 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:37.395311 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:37.395392 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:33:37.395764 (+ 372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:37.396391 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:37.397637 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:37.398017 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:37.398154 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:37.398210 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:37.398435 (+ 225us) server_negotiation.cc:299] Negotiation successful
0807 18:33:37.398532 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260807 18:33:37.399292 19089 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c121358a3dd441ff9162618b95625837" candidate_uuid: "a290fe9b958d48f0a540e728e58c4bc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f25e947f8c2c4fef895cfd78048211d7" is_pre_election: true
I20260807 18:33:37.399488 19235 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c121358a3dd441ff9162618b95625837" candidate_uuid: "a290fe9b958d48f0a540e728e58c4bc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a3774b03adf44d3b1475982c04da6aa" is_pre_election: true
I20260807 18:33:37.399699 19089 raft_consensus.cc:2454] T c121358a3dd441ff9162618b95625837 P f25e947f8c2c4fef895cfd78048211d7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a290fe9b958d48f0a540e728e58c4bc7 in term 0.
I20260807 18:33:37.399797 19235 raft_consensus.cc:2454] T c121358a3dd441ff9162618b95625837 P 6a3774b03adf44d3b1475982c04da6aa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a290fe9b958d48f0a540e728e58c4bc7 in term 0.
I20260807 18:33:37.400264 18869 leader_election.cc:302] T c121358a3dd441ff9162618b95625837 P a290fe9b958d48f0a540e728e58c4bc7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 6a3774b03adf44d3b1475982c04da6aa, a290fe9b958d48f0a540e728e58c4bc7, f25e947f8c2c4fef895cfd78048211d7; no voters:
I20260807 18:33:37.400586 19329 raft_consensus.cc:2781] T c121358a3dd441ff9162618b95625837 P a290fe9b958d48f0a540e728e58c4bc7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:33:37.400669 19329 raft_consensus.cc:482] T c121358a3dd441ff9162618b95625837 P a290fe9b958d48f0a540e728e58c4bc7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:33:37.400700 19329 raft_consensus.cc:3037] T c121358a3dd441ff9162618b95625837 P a290fe9b958d48f0a540e728e58c4bc7 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:33:37.401821 19329 raft_consensus.cc:504] T c121358a3dd441ff9162618b95625837 P a290fe9b958d48f0a540e728e58c4bc7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f25e947f8c2c4fef895cfd78048211d7" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 39193 } } peers { permanent_uuid: "6a3774b03adf44d3b1475982c04da6aa" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 42159 } } peers { permanent_uuid: "a290fe9b958d48f0a540e728e58c4bc7" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 33879 } }
I20260807 18:33:37.402187 19329 leader_election.cc:288] T c121358a3dd441ff9162618b95625837 P a290fe9b958d48f0a540e728e58c4bc7 [CANDIDATE]: Term 1 election: Requested vote from peers f25e947f8c2c4fef895cfd78048211d7 (127.11.14.66:39193), 6a3774b03adf44d3b1475982c04da6aa (127.11.14.67:42159)
I20260807 18:33:37.402670 19235 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c121358a3dd441ff9162618b95625837" candidate_uuid: "a290fe9b958d48f0a540e728e58c4bc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a3774b03adf44d3b1475982c04da6aa"
I20260807 18:33:37.402750 19089 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c121358a3dd441ff9162618b95625837" candidate_uuid: "a290fe9b958d48f0a540e728e58c4bc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f25e947f8c2c4fef895cfd78048211d7"
I20260807 18:33:37.402866 19235 raft_consensus.cc:3037] T c121358a3dd441ff9162618b95625837 P 6a3774b03adf44d3b1475982c04da6aa [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:33:37.402925 19089 raft_consensus.cc:3037] T c121358a3dd441ff9162618b95625837 P f25e947f8c2c4fef895cfd78048211d7 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:33:37.404480 19235 raft_consensus.cc:2454] T c121358a3dd441ff9162618b95625837 P 6a3774b03adf44d3b1475982c04da6aa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a290fe9b958d48f0a540e728e58c4bc7 in term 1.
I20260807 18:33:37.404480 19089 raft_consensus.cc:2454] T c121358a3dd441ff9162618b95625837 P f25e947f8c2c4fef895cfd78048211d7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a290fe9b958d48f0a540e728e58c4bc7 in term 1.
I20260807 18:33:37.404807 18869 leader_election.cc:302] T c121358a3dd441ff9162618b95625837 P a290fe9b958d48f0a540e728e58c4bc7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a3774b03adf44d3b1475982c04da6aa, a290fe9b958d48f0a540e728e58c4bc7; no voters:
I20260807 18:33:37.405076 19329 raft_consensus.cc:2781] T c121358a3dd441ff9162618b95625837 P a290fe9b958d48f0a540e728e58c4bc7 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:33:37.405316 19329 raft_consensus.cc:686] T c121358a3dd441ff9162618b95625837 P a290fe9b958d48f0a540e728e58c4bc7 [term 1 LEADER]: Becoming Leader. State: Replica: a290fe9b958d48f0a540e728e58c4bc7, State: Running, Role: LEADER
I20260807 18:33:37.405680 19329 consensus_queue.cc:237] T c121358a3dd441ff9162618b95625837 P a290fe9b958d48f0a540e728e58c4bc7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f25e947f8c2c4fef895cfd78048211d7" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 39193 } } peers { permanent_uuid: "6a3774b03adf44d3b1475982c04da6aa" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 42159 } } peers { permanent_uuid: "a290fe9b958d48f0a540e728e58c4bc7" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 33879 } }
I20260807 18:33:37.409741 17962 catalog_manager.cc:5495] T c121358a3dd441ff9162618b95625837 P a290fe9b958d48f0a540e728e58c4bc7 reported cstate change: term changed from 0 to 1, leader changed from <none> to a290fe9b958d48f0a540e728e58c4bc7 (127.11.14.65). New cstate: current_term: 1 leader_uuid: "a290fe9b958d48f0a540e728e58c4bc7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f25e947f8c2c4fef895cfd78048211d7" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 39193 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a3774b03adf44d3b1475982c04da6aa" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 42159 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a290fe9b958d48f0a540e728e58c4bc7" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 33879 } health_report { overall_health: HEALTHY } } }
I20260807 18:33:37.430956 19322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:37.426796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60098 (local address 127.11.14.65:33879)
0807 18:33:37.426955 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:37.426959 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:37.426974 (+ 15us) server_negotiation.cc:407] Connection header received
0807 18:33:37.427149 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:37.427151 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:37.427205 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:37.427294 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:33:37.427773 (+ 479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:37.428398 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:37.429533 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:37.429738 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:37.429820 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:37.430133 (+ 313us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:33:37.430555 (+ 422us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:33:37.430653 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:37.430784 (+ 131us) server_negotiation.cc:299] Negotiation successful
0807 18:33:37.430843 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260807 18:33:37.437008 19089 raft_consensus.cc:1261] T c121358a3dd441ff9162618b95625837 P f25e947f8c2c4fef895cfd78048211d7 [term 1 FOLLOWER]: Refusing update from remote peer a290fe9b958d48f0a540e728e58c4bc7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:33:37.437057 19235 raft_consensus.cc:1261] T c121358a3dd441ff9162618b95625837 P 6a3774b03adf44d3b1475982c04da6aa [term 1 FOLLOWER]: Refusing update from remote peer a290fe9b958d48f0a540e728e58c4bc7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:33:37.437836 19329 consensus_queue.cc:1035] T c121358a3dd441ff9162618b95625837 P a290fe9b958d48f0a540e728e58c4bc7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a3774b03adf44d3b1475982c04da6aa" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 42159 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:33:37.438047 19334 consensus_queue.cc:1035] T c121358a3dd441ff9162618b95625837 P a290fe9b958d48f0a540e728e58c4bc7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f25e947f8c2c4fef895cfd78048211d7" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 39193 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:33:37.450830 19340 mvcc.cc:205] Tried to move back new op lower bound from 7315978721014513664 to 7315978720896741376. Current Snapshot: MvccSnapshot[applied={T|T < 7315978721014513664}]
I20260807 18:33:37.451748 19339 mvcc.cc:205] Tried to move back new op lower bound from 7315978721014513664 to 7315978720896741376. Current Snapshot: MvccSnapshot[applied={T|T < 7315978721014513664 or (T in {7315978721014513664})}]
I20260807 18:33:37.455248 17962 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:32994:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:33:37.455422 17962 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:32994:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:33:37.460083 17962 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9b176ea138c9402a95ef84541f6cc5e3: Sending DeleteTablet for 3 replicas of tablet c121358a3dd441ff9162618b95625837
I20260807 18:33:37.460829 19068 tablet_service.cc:1514] Processing DeleteTablet for tablet c121358a3dd441ff9162618b95625837 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:33:37 UTC) from {username='oryx', principal='oryx/127.11.14.126@KRBTEST.COM'} at 127.0.0.1:36710
I20260807 18:33:37.461027 19214 tablet_service.cc:1514] Processing DeleteTablet for tablet c121358a3dd441ff9162618b95625837 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:33:37 UTC) from {username='oryx', principal='oryx/127.11.14.126@KRBTEST.COM'} at 127.0.0.1:40382
I20260807 18:33:37.461185 18919 tablet_service.cc:1514] Processing DeleteTablet for tablet c121358a3dd441ff9162618b95625837 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:33:37 UTC) from {username='oryx', principal='oryx/127.11.14.126@KRBTEST.COM'} at 127.0.0.1:60094
I20260807 18:33:37.461560 19347 tablet_replica.cc:330] T c121358a3dd441ff9162618b95625837 P f25e947f8c2c4fef895cfd78048211d7: stopping tablet replica
I20260807 18:33:37.461905 19347 raft_consensus.cc:2229] T c121358a3dd441ff9162618b95625837 P f25e947f8c2c4fef895cfd78048211d7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:33:37.462240 19347 raft_consensus.cc:2258] T c121358a3dd441ff9162618b95625837 P f25e947f8c2c4fef895cfd78048211d7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:33:37.463622 11321 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkz56XN/build/debug/bin/kudu with pid 17928
I20260807 18:33:37.466583 19347 ts_tablet_manager.cc:1905] T c121358a3dd441ff9162618b95625837 P f25e947f8c2c4fef895cfd78048211d7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:33:37.470649 19347 ts_tablet_manager.cc:1918] T c121358a3dd441ff9162618b95625837 P f25e947f8c2c4fef895cfd78048211d7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:33:37.470757 19347 log.cc:1196] T c121358a3dd441ff9162618b95625837 P f25e947f8c2c4fef895cfd78048211d7: Deleting WAL directory at /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/minicluster-data/ts-1/wal/wals/c121358a3dd441ff9162618b95625837
I20260807 18:33:37.471117 19347 ts_tablet_manager.cc:1939] T c121358a3dd441ff9162618b95625837 P f25e947f8c2c4fef895cfd78048211d7: Deleting consensus metadata
I20260807 18:33:37.471872 19349 tablet_replica.cc:330] T c121358a3dd441ff9162618b95625837 P a290fe9b958d48f0a540e728e58c4bc7: stopping tablet replica
I20260807 18:33:37.472293 19349 raft_consensus.cc:2229] T c121358a3dd441ff9162618b95625837 P a290fe9b958d48f0a540e728e58c4bc7 [term 1 LEADER]: Raft consensus shutting down.
I20260807 18:33:37.472738 19349 raft_consensus.cc:2258] T c121358a3dd441ff9162618b95625837 P a290fe9b958d48f0a540e728e58c4bc7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:33:37.473197 19348 tablet_replica.cc:330] T c121358a3dd441ff9162618b95625837 P 6a3774b03adf44d3b1475982c04da6aa: stopping tablet replica
I20260807 18:33:37.473564 19348 raft_consensus.cc:2229] T c121358a3dd441ff9162618b95625837 P 6a3774b03adf44d3b1475982c04da6aa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:33:37.473935 19348 raft_consensus.cc:2258] T c121358a3dd441ff9162618b95625837 P 6a3774b03adf44d3b1475982c04da6aa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:33:37.474434 19349 ts_tablet_manager.cc:1905] T c121358a3dd441ff9162618b95625837 P a290fe9b958d48f0a540e728e58c4bc7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:33:37.476034 19348 ts_tablet_manager.cc:1905] T c121358a3dd441ff9162618b95625837 P 6a3774b03adf44d3b1475982c04da6aa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:33:37.477195 19349 ts_tablet_manager.cc:1918] T c121358a3dd441ff9162618b95625837 P a290fe9b958d48f0a540e728e58c4bc7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:33:37.477296 19349 log.cc:1196] T c121358a3dd441ff9162618b95625837 P a290fe9b958d48f0a540e728e58c4bc7: Deleting WAL directory at /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/minicluster-data/ts-0/wal/wals/c121358a3dd441ff9162618b95625837
W20260807 18:33:37.477497 19018 connection.cc:535] server connection from 127.0.0.1:36710 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260807 18:33:37.477626 19349 ts_tablet_manager.cc:1939] T c121358a3dd441ff9162618b95625837 P a290fe9b958d48f0a540e728e58c4bc7: Deleting consensus metadata
W20260807 18:33:37.478016 18585 connection.cc:535] server connection from 127.0.0.1:54266 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260807 18:33:37.478842 18868 connection.cc:441] server connection from 127.0.0.1:60094 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:60094 (request call id 1) could send its response
W20260807 18:33:37.479270 18781 connection.cc:535] server connection from 127.0.0.1:44600 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260807 18:33:37.480008 19348 ts_tablet_manager.cc:1918] T c121358a3dd441ff9162618b95625837 P 6a3774b03adf44d3b1475982c04da6aa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:33:37.480149 19348 log.cc:1196] T c121358a3dd441ff9162618b95625837 P 6a3774b03adf44d3b1475982c04da6aa: Deleting WAL directory at /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/minicluster-data/ts-2/wal/wals/c121358a3dd441ff9162618b95625837
I20260807 18:33:37.480553 19348 ts_tablet_manager.cc:1939] T c121358a3dd441ff9162618b95625837 P 6a3774b03adf44d3b1475982c04da6aa: Deleting consensus metadata
W20260807 18:33:37.482709 19168 connection.cc:441] server connection from 127.0.0.1:40382 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:40382 (request call id 1) could send its response
I20260807 18:33:37.495785 18995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:37.484509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44620 (local address 127.11.14.124:36503)
0807 18:33:37.484677 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:37.484683 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:37.486167 (+ 1484us) server_negotiation.cc:407] Connection header received
0807 18:33:37.486349 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:37.486352 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:37.486402 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:37.486454 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:33:37.487316 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:37.488105 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:37.489955 (+ 1850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:37.490299 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:37.490377 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:37.492174 (+ 1797us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:33:37.492178 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:33:37.492181 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:33:37.492209 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:33:37.493919 (+ 1710us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:37.494410 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:37.494413 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:37.494415 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:37.494496 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:37.494779 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:37.494782 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:37.494802 (+ 20us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:37.495011 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:33:37.495112 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:37.495467 (+ 355us) server_negotiation.cc:299] Negotiation successful
0807 18:33:37.495617 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260807 18:33:37.495841 18730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:37.483824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54280 (local address 127.11.14.125:39483)
0807 18:33:37.483970 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:37.483976 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:37.484329 (+ 353us) server_negotiation.cc:407] Connection header received
0807 18:33:37.484551 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:37.484555 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:37.484612 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:37.484676 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:33:37.485733 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:37.486574 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:37.488289 (+ 1715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:37.488670 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:37.488770 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:37.490916 (+ 2146us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:33:37.490923 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:33:37.490930 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:33:37.490967 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:33:37.493294 (+ 2327us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:37.493961 (+ 667us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:37.493965 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:37.493967 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:37.494064 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:37.494468 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:37.494473 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:37.494475 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:37.494719 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:33:37.494853 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:37.495471 (+ 618us) server_negotiation.cc:299] Negotiation successful
0807 18:33:37.495629 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260807 18:33:37.496289 19301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:37.463741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54276 (local address 127.11.14.125:39483)
0807 18:33:37.465512 (+ 1771us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:37.465520 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:37.479907 (+ 14387us) server_negotiation.cc:407] Connection header received
0807 18:33:37.480183 (+ 276us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:37.480186 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:37.480243 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:37.480298 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:33:37.481464 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:37.482311 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:37.483643 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:37.484017 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:37.484091 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:37.486018 (+ 1927us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:33:37.486023 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:33:37.486026 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:33:37.486057 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:33:37.488316 (+ 2259us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:37.489133 (+ 817us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:37.489138 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:37.489140 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:37.489286 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:37.490041 (+ 755us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:37.490049 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:37.490053 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:37.490358 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:33:37.490522 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:37.491380 (+ 858us) server_negotiation.cc:299] Negotiation successful
0807 18:33:37.491544 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1629}
I20260807 18:33:37.501700 19303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:37.466133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44614 (local address 127.11.14.124:36503)
0807 18:33:37.466263 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:37.466268 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:37.492059 (+ 25791us) server_negotiation.cc:407] Connection header received
0807 18:33:37.492127 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:37.492132 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:37.492202 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:37.492298 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:33:37.493247 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:37.494075 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:37.495281 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:37.495710 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:37.495826 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:37.498355 (+ 2529us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:33:37.498359 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:33:37.498362 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:33:37.498387 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:33:37.500071 (+ 1684us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:37.500511 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:37.500515 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:37.500517 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:37.500603 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:37.500896 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:37.500900 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:37.500902 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:37.501095 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:33:37.501217 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:37.501435 (+ 218us) server_negotiation.cc:299] Negotiation successful
0807 18:33:37.501556 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260807 18:33:38.455042 19358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:38.454510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:45409 (local address 127.11.14.66:36219)
0807 18:33:38.454801 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:38.454923 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.126:45409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":109,"threads_started":1}
I20260807 18:33:38.455281 19359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:38.454820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:45409 (local address 127.11.14.65:38695)
0807 18:33:38.455073 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:38.455186 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.126:45409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":94,"threads_started":1}
W20260807 18:33:38.455494 19139 heartbeater.cc:644] Failed to heartbeat to 127.11.14.126:45409 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.11.14.126:45409: connect: Connection refused (error 111)
W20260807 18:33:38.455803 18992 heartbeater.cc:644] Failed to heartbeat to 127.11.14.126:45409 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.11.14.126:45409: connect: Connection refused (error 111)
I20260807 18:33:38.456197 19360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:38.455723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:45409 (local address 127.11.14.67:57693)
0807 18:33:38.456006 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:38.456093 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.126:45409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":55,"threads_started":1}
W20260807 18:33:38.456475 19288 heartbeater.cc:644] Failed to heartbeat to 127.11.14.126:45409 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.11.14.126:45409: connect: Connection refused (error 111)
I20260807 18:33:38.967335 19361 raft_consensus.cc:482] T 00000000000000000000000000000000 P ab43b0280a964c18a81ab0531a0a2a10 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9b176ea138c9402a95ef84541f6cc5e3)
I20260807 18:33:38.967459 19361 raft_consensus.cc:504] T 00000000000000000000000000000000 P ab43b0280a964c18a81ab0531a0a2a10 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "9b176ea138c9402a95ef84541f6cc5e3" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 45409 } } peers { permanent_uuid: "1abe9cee5c844198bbc7d32fdfae4374" member_type: VOTER last_known_addr { host: "127.11.14.125" port: 39483 } attrs { promote: false } } peers { permanent_uuid: "ab43b0280a964c18a81ab0531a0a2a10" member_type: VOTER last_known_addr { host: "127.11.14.124" port: 36503 } attrs { promote: false } }
I20260807 18:33:38.967957 19361 leader_election.cc:288] T 00000000000000000000000000000000 P ab43b0280a964c18a81ab0531a0a2a10 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9b176ea138c9402a95ef84541f6cc5e3 (127.11.14.126:45409), 1abe9cee5c844198bbc7d32fdfae4374 (127.11.14.125:39483)
I20260807 18:33:38.968762 19362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:38.968221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:45409 (local address 127.0.0.1:42294)
0807 18:33:38.968545 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:38.968643 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.126:45409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":70,"threads_started":1}
I20260807 18:33:38.968739 19364 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1abe9cee5c844198bbc7d32fdfae4374 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9b176ea138c9402a95ef84541f6cc5e3)
W20260807 18:33:38.968993 18780 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.11.14.126:45409: connect: Connection refused (error 111)
I20260807 18:33:38.969046 19364 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1abe9cee5c844198bbc7d32fdfae4374 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "9b176ea138c9402a95ef84541f6cc5e3" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 45409 } } peers { permanent_uuid: "1abe9cee5c844198bbc7d32fdfae4374" member_type: VOTER last_known_addr { host: "127.11.14.125" port: 39483 } attrs { promote: false } } peers { permanent_uuid: "ab43b0280a964c18a81ab0531a0a2a10" member_type: VOTER last_known_addr { host: "127.11.14.124" port: 36503 } attrs { promote: false } }
I20260807 18:33:38.969952 19362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:38.969670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:45409 (local address 127.0.0.1:42300)
0807 18:33:38.969834 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:38.969886 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.126:45409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
W20260807 18:33:38.970065 18780 leader_election.cc:334] T 00000000000000000000000000000000 P ab43b0280a964c18a81ab0531a0a2a10 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 9b176ea138c9402a95ef84541f6cc5e3 (127.11.14.126:45409): Network error: Client connection negotiation failed: client connection to 127.11.14.126:45409: connect: Connection refused (error 111)
I20260807 18:33:38.970389 19364 leader_election.cc:288] T 00000000000000000000000000000000 P 1abe9cee5c844198bbc7d32fdfae4374 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9b176ea138c9402a95ef84541f6cc5e3 (127.11.14.126:45409), ab43b0280a964c18a81ab0531a0a2a10 (127.11.14.124:36503)
I20260807 18:33:38.971000 19367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:38.970445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:45409 (local address 127.0.0.1:42312)
0807 18:33:38.970808 (+ 363us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:38.970894 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.126:45409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":295,"thread_start_us":71,"threads_started":1}
W20260807 18:33:38.971200 18585 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.11.14.126:45409: connect: Connection refused (error 111)
I20260807 18:33:38.972379 19367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:38.972086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:45409 (local address 127.0.0.1:42320)
0807 18:33:38.972261 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:38.972313 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.126:45409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91}
W20260807 18:33:38.972504 18585 leader_election.cc:334] T 00000000000000000000000000000000 P 1abe9cee5c844198bbc7d32fdfae4374 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 9b176ea138c9402a95ef84541f6cc5e3 (127.11.14.126:45409): Network error: Client connection negotiation failed: client connection to 127.11.14.126:45409: connect: Connection refused (error 111)
I20260807 18:33:38.973268 19365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:38.968595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57380 (local address 127.11.14.125:39483)
0807 18:33:38.968911 (+ 316us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:38.968916 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:38.968932 (+ 16us) server_negotiation.cc:407] Connection header received
0807 18:33:38.968985 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:38.968991 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:38.969042 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:38.969140 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:33:38.969733 (+ 593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:38.970520 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:38.972158 (+ 1638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:38.972716 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:38.972853 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:38.972918 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:38.973085 (+ 167us) server_negotiation.cc:299] Negotiation successful
0807 18:33:38.973160 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":243,"thread_start_us":96,"threads_started":1}
I20260807 18:33:38.973268 19363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:38.968277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.125:39483 (local address 127.0.0.1:57380)
0807 18:33:38.968571 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:38.968606 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:38.968734 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:38.969326 (+ 592us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:38.969331 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:38.969358 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:33:38.969493 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:38.969501 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:38.970665 (+ 1164us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:38.970670 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:38.972031 (+ 1361us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:38.972041 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:38.972844 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:38.972848 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:38.972943 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:38.972966 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 18:33:38.973069 (+ 103us) client_negotiation.cc:240] Negotiation successful
0807 18:33:38.973144 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":70,"threads_started":1}
I20260807 18:33:38.973847 18611 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ab43b0280a964c18a81ab0531a0a2a10" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "1abe9cee5c844198bbc7d32fdfae4374" is_pre_election: true
I20260807 18:33:38.973999 18611 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 1abe9cee5c844198bbc7d32fdfae4374 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ab43b0280a964c18a81ab0531a0a2a10 in term 1.
I20260807 18:33:38.974364 18779 leader_election.cc:302] T 00000000000000000000000000000000 P ab43b0280a964c18a81ab0531a0a2a10 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1abe9cee5c844198bbc7d32fdfae4374, ab43b0280a964c18a81ab0531a0a2a10; no voters: 9b176ea138c9402a95ef84541f6cc5e3
I20260807 18:33:38.974659 19361 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ab43b0280a964c18a81ab0531a0a2a10 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 18:33:38.974772 19361 raft_consensus.cc:482] T 00000000000000000000000000000000 P ab43b0280a964c18a81ab0531a0a2a10 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9b176ea138c9402a95ef84541f6cc5e3)
I20260807 18:33:38.974845 19361 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ab43b0280a964c18a81ab0531a0a2a10 [term 1 FOLLOWER]: Advancing to term 2
I20260807 18:33:38.975205 19368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:38.970620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.124:36503 (local address 127.0.0.1:50922)
0807 18:33:38.971008 (+ 388us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:38.971022 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:38.971110 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:38.971415 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:38.971418 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:38.971437 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:33:38.971523 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:38.971528 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:38.972646 (+ 1118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:38.972650 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:38.973776 (+ 1126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:38.973793 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:38.974819 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:38.974824 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:38.974966 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:38.974998 (+ 32us) client_negotiation.cc:769] Sending connection context
0807 18:33:38.975064 (+ 66us) client_negotiation.cc:240] Negotiation successful
0807 18:33:38.975118 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":323,"thread_start_us":93,"threads_started":1}
I20260807 18:33:38.975359 19369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:38.970736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50922 (local address 127.11.14.124:36503)
0807 18:33:38.971031 (+ 295us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:38.971044 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:38.971157 (+ 113us) server_negotiation.cc:407] Connection header received
0807 18:33:38.971214 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:38.971221 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:38.971280 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:38.971385 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:33:38.971698 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:38.972458 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:38.973937 (+ 1479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:38.974651 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:38.974800 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:38.974885 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:38.975128 (+ 243us) server_negotiation.cc:299] Negotiation successful
0807 18:33:38.975216 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":218,"thread_start_us":110,"threads_started":1}
I20260807 18:33:38.975840 18807 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1abe9cee5c844198bbc7d32fdfae4374" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "ab43b0280a964c18a81ab0531a0a2a10" is_pre_election: true
I20260807 18:33:38.976392 19361 raft_consensus.cc:504] T 00000000000000000000000000000000 P ab43b0280a964c18a81ab0531a0a2a10 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "9b176ea138c9402a95ef84541f6cc5e3" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 45409 } } peers { permanent_uuid: "1abe9cee5c844198bbc7d32fdfae4374" member_type: VOTER last_known_addr { host: "127.11.14.125" port: 39483 } attrs { promote: false } } peers { permanent_uuid: "ab43b0280a964c18a81ab0531a0a2a10" member_type: VOTER last_known_addr { host: "127.11.14.124" port: 36503 } attrs { promote: false } }
I20260807 18:33:38.976713 18807 raft_consensus.cc:2379] T 00000000000000000000000000000000 P ab43b0280a964c18a81ab0531a0a2a10 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1abe9cee5c844198bbc7d32fdfae4374 in current term 2: Already voted for candidate ab43b0280a964c18a81ab0531a0a2a10 in this term.
I20260807 18:33:38.976926 19361 leader_election.cc:288] T 00000000000000000000000000000000 P ab43b0280a964c18a81ab0531a0a2a10 [CANDIDATE]: Term 2 election: Requested vote from peers 9b176ea138c9402a95ef84541f6cc5e3 (127.11.14.126:45409), 1abe9cee5c844198bbc7d32fdfae4374 (127.11.14.125:39483)
I20260807 18:33:38.977169 18584 leader_election.cc:302] T 00000000000000000000000000000000 P 1abe9cee5c844198bbc7d32fdfae4374 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1abe9cee5c844198bbc7d32fdfae4374; no voters: 9b176ea138c9402a95ef84541f6cc5e3, ab43b0280a964c18a81ab0531a0a2a10
I20260807 18:33:38.977582 19363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:38.977326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:45409 (local address 127.0.0.1:42334)
0807 18:33:38.977455 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:38.977515 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.126:45409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260807 18:33:38.977579 19364 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1abe9cee5c844198bbc7d32fdfae4374 [term 1 FOLLOWER]: Advancing to term 2
I20260807 18:33:38.978294 19363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:38.978046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:45409 (local address 127.0.0.1:42336)
0807 18:33:38.978177 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:38.978240 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.126:45409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260807 18:33:38.978431 18780 leader_election.cc:334] T 00000000000000000000000000000000 P ab43b0280a964c18a81ab0531a0a2a10 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 9b176ea138c9402a95ef84541f6cc5e3 (127.11.14.126:45409): Network error: Client connection negotiation failed: client connection to 127.11.14.126:45409: connect: Connection refused (error 111)
I20260807 18:33:38.978783 19364 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 1abe9cee5c844198bbc7d32fdfae4374 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 18:33:38.979000 18611 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ab43b0280a964c18a81ab0531a0a2a10" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "1abe9cee5c844198bbc7d32fdfae4374"
I20260807 18:33:38.979949 18611 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 1abe9cee5c844198bbc7d32fdfae4374 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ab43b0280a964c18a81ab0531a0a2a10 in term 2.
I20260807 18:33:38.980270 18779 leader_election.cc:302] T 00000000000000000000000000000000 P ab43b0280a964c18a81ab0531a0a2a10 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1abe9cee5c844198bbc7d32fdfae4374, ab43b0280a964c18a81ab0531a0a2a10; no voters: 9b176ea138c9402a95ef84541f6cc5e3
I20260807 18:33:38.980502 19361 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ab43b0280a964c18a81ab0531a0a2a10 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 18:33:38.980788 19361 raft_consensus.cc:686] T 00000000000000000000000000000000 P ab43b0280a964c18a81ab0531a0a2a10 [term 2 LEADER]: Becoming Leader. State: Replica: ab43b0280a964c18a81ab0531a0a2a10, State: Running, Role: LEADER
I20260807 18:33:38.981086 19361 consensus_queue.cc:237] T 00000000000000000000000000000000 P ab43b0280a964c18a81ab0531a0a2a10 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "9b176ea138c9402a95ef84541f6cc5e3" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 45409 } } peers { permanent_uuid: "1abe9cee5c844198bbc7d32fdfae4374" member_type: VOTER last_known_addr { host: "127.11.14.125" port: 39483 } attrs { promote: false } } peers { permanent_uuid: "ab43b0280a964c18a81ab0531a0a2a10" member_type: VOTER last_known_addr { host: "127.11.14.124" port: 36503 } attrs { promote: false } }
I20260807 18:33:38.982697 19371 sys_catalog.cc:455] T 00000000000000000000000000000000 P ab43b0280a964c18a81ab0531a0a2a10 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ab43b0280a964c18a81ab0531a0a2a10. Latest consensus state: current_term: 2 leader_uuid: "ab43b0280a964c18a81ab0531a0a2a10" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "9b176ea138c9402a95ef84541f6cc5e3" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 45409 } } peers { permanent_uuid: "1abe9cee5c844198bbc7d32fdfae4374" member_type: VOTER last_known_addr { host: "127.11.14.125" port: 39483 } attrs { promote: false } } peers { permanent_uuid: "ab43b0280a964c18a81ab0531a0a2a10" member_type: VOTER last_known_addr { host: "127.11.14.124" port: 36503 } attrs { promote: false } } }
I20260807 18:33:38.982820 19371 sys_catalog.cc:458] T 00000000000000000000000000000000 P ab43b0280a964c18a81ab0531a0a2a10 [sys.catalog]: This master's current role is: LEADER
I20260807 18:33:38.983199 19373 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:33:38.985775 19373 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:33:38.986303 19373 catalog_manager.cc:1259] Loaded cluster ID: b84b754b281a4b06b86e0ce85d0fbbe8
I20260807 18:33:38.986383 19373 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:33:38.987025 19373 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:33:38.987455 19373 catalog_manager.cc:5879] T 00000000000000000000000000000000 P ab43b0280a964c18a81ab0531a0a2a10: Loaded TSK: 0
I20260807 18:33:38.988238 19373 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:33:39.028740 18795 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44620:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 18:33:39.032383 18795 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 18:33:39.036016 19363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:39.035667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:45409 (local address 127.0.0.1:42356)
0807 18:33:39.035824 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:39.035902 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.126:45409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260807 18:33:39.036240 18611 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1abe9cee5c844198bbc7d32fdfae4374 [term 2 FOLLOWER]: Refusing update from remote peer ab43b0280a964c18a81ab0531a0a2a10: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260807 18:33:39.036897 19363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:39.036572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:45409 (local address 127.0.0.1:42360)
0807 18:33:39.036720 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:39.036788 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.126:45409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260807 18:33:39.036952 19371 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ab43b0280a964c18a81ab0531a0a2a10 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1abe9cee5c844198bbc7d32fdfae4374" member_type: VOTER last_known_addr { host: "127.11.14.125" port: 39483 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260807 18:33:39.037158 18780 consensus_peers.cc:487] T 00000000000000000000000000000000 P ab43b0280a964c18a81ab0531a0a2a10 -> Peer 9b176ea138c9402a95ef84541f6cc5e3 (127.11.14.126:45409): Couldn't send request to peer 9b176ea138c9402a95ef84541f6cc5e3. Status: Network error: Client connection negotiation failed: client connection to 127.11.14.126:45409: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260807 18:33:39.038546 19364 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1abe9cee5c844198bbc7d32fdfae4374 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ab43b0280a964c18a81ab0531a0a2a10. Latest consensus state: current_term: 2 leader_uuid: "ab43b0280a964c18a81ab0531a0a2a10" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "9b176ea138c9402a95ef84541f6cc5e3" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 45409 } } peers { permanent_uuid: "1abe9cee5c844198bbc7d32fdfae4374" member_type: VOTER last_known_addr { host: "127.11.14.125" port: 39483 } attrs { promote: false } } peers { permanent_uuid: "ab43b0280a964c18a81ab0531a0a2a10" member_type: VOTER last_known_addr { host: "127.11.14.124" port: 36503 } attrs { promote: false } } }
I20260807 18:33:39.038714 19364 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1abe9cee5c844198bbc7d32fdfae4374 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 18:33:39.039886 19371 sys_catalog.cc:455] T 00000000000000000000000000000000 P ab43b0280a964c18a81ab0531a0a2a10 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "ab43b0280a964c18a81ab0531a0a2a10" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "9b176ea138c9402a95ef84541f6cc5e3" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 45409 } } peers { permanent_uuid: "1abe9cee5c844198bbc7d32fdfae4374" member_type: VOTER last_known_addr { host: "127.11.14.125" port: 39483 } attrs { promote: false } } peers { permanent_uuid: "ab43b0280a964c18a81ab0531a0a2a10" member_type: VOTER last_known_addr { host: "127.11.14.124" port: 36503 } attrs { promote: false } } }
I20260807 18:33:39.040045 19371 sys_catalog.cc:458] T 00000000000000000000000000000000 P ab43b0280a964c18a81ab0531a0a2a10 [sys.catalog]: This master's current role is: LEADER
I20260807 18:33:39.040611 19364 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1abe9cee5c844198bbc7d32fdfae4374 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "ab43b0280a964c18a81ab0531a0a2a10" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "9b176ea138c9402a95ef84541f6cc5e3" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 45409 } } peers { permanent_uuid: "1abe9cee5c844198bbc7d32fdfae4374" member_type: VOTER last_known_addr { host: "127.11.14.125" port: 39483 } attrs { promote: false } } peers { permanent_uuid: "ab43b0280a964c18a81ab0531a0a2a10" member_type: VOTER last_known_addr { host: "127.11.14.124" port: 36503 } attrs { promote: false } } }
I20260807 18:33:39.040742 19364 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1abe9cee5c844198bbc7d32fdfae4374 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 18:33:39.042577 19375 mvcc.cc:205] Tried to move back new op lower bound from 7315978727565582336 to 7315978727349657600. Current Snapshot: MvccSnapshot[applied={T|T < 7315978727565582336 or (T in {7315978727565582336})}]
I20260807 18:33:39.051883 19362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:39.046470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.67:42159 (local address 127.0.0.1:36510)
0807 18:33:39.046601 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:39.046619 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:39.046715 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:39.047502 (+ 787us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:39.047505 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:39.047523 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:33:39.047715 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:39.047722 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:39.048917 (+ 1195us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:39.048920 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:39.050216 (+ 1296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:39.050228 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:39.051280 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:39.051285 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:39.051389 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:39.051434 (+ 45us) client_negotiation.cc:769] Sending connection context
0807 18:33:39.051497 (+ 63us) client_negotiation.cc:240] Negotiation successful
0807 18:33:39.051659 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56,"mutex_wait_us":86}
I20260807 18:33:39.051898 19363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:39.046205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.66:39193 (local address 127.0.0.1:35412)
0807 18:33:39.046317 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:39.046341 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:39.046442 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:39.047675 (+ 1233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:39.047678 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:39.047692 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:33:39.047757 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:39.047765 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:39.048789 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:39.048795 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:39.050216 (+ 1421us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:39.050228 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:39.051183 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:39.051187 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:39.051549 (+ 362us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:39.051598 (+ 49us) client_negotiation.cc:769] Sending connection context
0807 18:33:39.051649 (+ 51us) client_negotiation.cc:240] Negotiation successful
0807 18:33:39.051729 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31,"mutex_wait_us":91}
I20260807 18:33:39.052291 19380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:39.046771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35412 (local address 127.11.14.66:39193)
0807 18:33:39.047210 (+ 439us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:39.047217 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:39.047239 (+ 22us) server_negotiation.cc:407] Connection header received
0807 18:33:39.047305 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:39.047309 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:39.047364 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:39.047480 (+ 116us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:33:39.047920 (+ 440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:39.048690 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:39.050379 (+ 1689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:39.051050 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:39.051969 (+ 919us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:39.052036 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:39.052074 (+ 38us) server_negotiation.cc:299] Negotiation successful
0807 18:33:39.052145 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":345,"thread_start_us":75,"threads_started":1}
I20260807 18:33:39.052295 19384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:39.046916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36510 (local address 127.11.14.67:42159)
0807 18:33:39.047220 (+ 304us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:39.047227 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:39.047243 (+ 16us) server_negotiation.cc:407] Connection header received
0807 18:33:39.047305 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:39.047308 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:39.047364 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:39.047480 (+ 116us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:33:39.047871 (+ 391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:39.048740 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:39.050371 (+ 1631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:39.051116 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:39.051835 (+ 719us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:39.051943 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:39.051995 (+ 52us) server_negotiation.cc:299] Negotiation successful
0807 18:33:39.052096 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":224,"thread_start_us":96,"threads_started":1}
I20260807 18:33:39.052938 19214 tablet_service.cc:1467] Processing CreateTablet for tablet d6e12785b29a4efc823770a057a8de7c (DEFAULT_TABLE table=test-table [id=10fa4c5ee15d4d1f8e3fa3ed78a46dd7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:33:39.053041 19383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:39.046930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60380 (local address 127.11.14.65:33879)
0807 18:33:39.047178 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:39.047182 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:39.048029 (+ 847us) server_negotiation.cc:407] Connection header received
0807 18:33:39.048093 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:39.048096 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:39.048148 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:39.048224 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:33:39.048843 (+ 619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:39.049469 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:39.050799 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:39.051450 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:39.051551 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:39.051646 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:39.052253 (+ 607us) server_negotiation.cc:299] Negotiation successful
0807 18:33:39.052353 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":171,"thread_start_us":82,"threads_started":1}
I20260807 18:33:39.053215 19382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:39.046817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.65:33879 (local address 127.0.0.1:60380)
0807 18:33:39.047756 (+ 939us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:39.047770 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:39.047926 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:39.048533 (+ 607us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:39.048536 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:39.048554 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:33:39.048666 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:39.048673 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:39.049611 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:39.049616 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:39.050637 (+ 1021us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:39.050650 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:39.051702 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:39.051706 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:39.051792 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:39.051813 (+ 21us) client_negotiation.cc:769] Sending connection context
0807 18:33:39.051893 (+ 80us) client_negotiation.cc:240] Negotiation successful
0807 18:33:39.051967 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":875,"mutex_wait_us":4,"thread_start_us":106,"threads_started":1}
I20260807 18:33:39.053288 19214 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d6e12785b29a4efc823770a057a8de7c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:33:39.053200 19068 tablet_service.cc:1467] Processing CreateTablet for tablet d6e12785b29a4efc823770a057a8de7c (DEFAULT_TABLE table=test-table [id=10fa4c5ee15d4d1f8e3fa3ed78a46dd7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:33:39.053537 19068 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d6e12785b29a4efc823770a057a8de7c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:33:39.053742 18919 tablet_service.cc:1467] Processing CreateTablet for tablet d6e12785b29a4efc823770a057a8de7c (DEFAULT_TABLE table=test-table [id=10fa4c5ee15d4d1f8e3fa3ed78a46dd7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:33:39.054014 18919 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d6e12785b29a4efc823770a057a8de7c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:33:39.056188 19385 tablet_bootstrap.cc:492] T d6e12785b29a4efc823770a057a8de7c P 6a3774b03adf44d3b1475982c04da6aa: Bootstrap starting.
I20260807 18:33:39.056684 19386 tablet_bootstrap.cc:492] T d6e12785b29a4efc823770a057a8de7c P f25e947f8c2c4fef895cfd78048211d7: Bootstrap starting.
I20260807 18:33:39.057415 19387 tablet_bootstrap.cc:492] T d6e12785b29a4efc823770a057a8de7c P a290fe9b958d48f0a540e728e58c4bc7: Bootstrap starting.
I20260807 18:33:39.057655 19385 tablet_bootstrap.cc:654] T d6e12785b29a4efc823770a057a8de7c P 6a3774b03adf44d3b1475982c04da6aa: Neither blocks nor log segments found. Creating new log.
I20260807 18:33:39.057745 19386 tablet_bootstrap.cc:654] T d6e12785b29a4efc823770a057a8de7c P f25e947f8c2c4fef895cfd78048211d7: Neither blocks nor log segments found. Creating new log.
I20260807 18:33:39.058626 19387 tablet_bootstrap.cc:654] T d6e12785b29a4efc823770a057a8de7c P a290fe9b958d48f0a540e728e58c4bc7: Neither blocks nor log segments found. Creating new log.
I20260807 18:33:39.060019 19387 tablet_bootstrap.cc:492] T d6e12785b29a4efc823770a057a8de7c P a290fe9b958d48f0a540e728e58c4bc7: No bootstrap required, opened a new log
I20260807 18:33:39.060019 19385 tablet_bootstrap.cc:492] T d6e12785b29a4efc823770a057a8de7c P 6a3774b03adf44d3b1475982c04da6aa: No bootstrap required, opened a new log
I20260807 18:33:39.060117 19387 ts_tablet_manager.cc:1397] T d6e12785b29a4efc823770a057a8de7c P a290fe9b958d48f0a540e728e58c4bc7: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260807 18:33:39.060117 19385 ts_tablet_manager.cc:1397] T d6e12785b29a4efc823770a057a8de7c P 6a3774b03adf44d3b1475982c04da6aa: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260807 18:33:39.060410 19386 tablet_bootstrap.cc:492] T d6e12785b29a4efc823770a057a8de7c P f25e947f8c2c4fef895cfd78048211d7: No bootstrap required, opened a new log
I20260807 18:33:39.060536 19386 ts_tablet_manager.cc:1397] T d6e12785b29a4efc823770a057a8de7c P f25e947f8c2c4fef895cfd78048211d7: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260807 18:33:39.060537 19387 raft_consensus.cc:348] T d6e12785b29a4efc823770a057a8de7c P a290fe9b958d48f0a540e728e58c4bc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f25e947f8c2c4fef895cfd78048211d7" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 39193 } } peers { permanent_uuid: "6a3774b03adf44d3b1475982c04da6aa" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 42159 } } peers { permanent_uuid: "a290fe9b958d48f0a540e728e58c4bc7" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 33879 } }
I20260807 18:33:39.060537 19385 raft_consensus.cc:348] T d6e12785b29a4efc823770a057a8de7c P 6a3774b03adf44d3b1475982c04da6aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f25e947f8c2c4fef895cfd78048211d7" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 39193 } } peers { permanent_uuid: "6a3774b03adf44d3b1475982c04da6aa" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 42159 } } peers { permanent_uuid: "a290fe9b958d48f0a540e728e58c4bc7" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 33879 } }
I20260807 18:33:39.060662 19387 raft_consensus.cc:374] T d6e12785b29a4efc823770a057a8de7c P a290fe9b958d48f0a540e728e58c4bc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:33:39.060662 19385 raft_consensus.cc:374] T d6e12785b29a4efc823770a057a8de7c P 6a3774b03adf44d3b1475982c04da6aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:33:39.060702 19385 raft_consensus.cc:729] T d6e12785b29a4efc823770a057a8de7c P 6a3774b03adf44d3b1475982c04da6aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a3774b03adf44d3b1475982c04da6aa, State: Initialized, Role: FOLLOWER
I20260807 18:33:39.060702 19387 raft_consensus.cc:729] T d6e12785b29a4efc823770a057a8de7c P a290fe9b958d48f0a540e728e58c4bc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a290fe9b958d48f0a540e728e58c4bc7, State: Initialized, Role: FOLLOWER
I20260807 18:33:39.060837 19387 consensus_queue.cc:260] T d6e12785b29a4efc823770a057a8de7c P a290fe9b958d48f0a540e728e58c4bc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f25e947f8c2c4fef895cfd78048211d7" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 39193 } } peers { permanent_uuid: "6a3774b03adf44d3b1475982c04da6aa" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 42159 } } peers { permanent_uuid: "a290fe9b958d48f0a540e728e58c4bc7" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 33879 } }
I20260807 18:33:39.060917 19385 consensus_queue.cc:260] T d6e12785b29a4efc823770a057a8de7c P 6a3774b03adf44d3b1475982c04da6aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f25e947f8c2c4fef895cfd78048211d7" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 39193 } } peers { permanent_uuid: "6a3774b03adf44d3b1475982c04da6aa" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 42159 } } peers { permanent_uuid: "a290fe9b958d48f0a540e728e58c4bc7" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 33879 } }
I20260807 18:33:39.061201 19387 ts_tablet_manager.cc:1428] T d6e12785b29a4efc823770a057a8de7c P a290fe9b958d48f0a540e728e58c4bc7: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 18:33:39.061143 19386 raft_consensus.cc:348] T d6e12785b29a4efc823770a057a8de7c P f25e947f8c2c4fef895cfd78048211d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f25e947f8c2c4fef895cfd78048211d7" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 39193 } } peers { permanent_uuid: "6a3774b03adf44d3b1475982c04da6aa" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 42159 } } peers { permanent_uuid: "a290fe9b958d48f0a540e728e58c4bc7" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 33879 } }
I20260807 18:33:39.061262 19385 ts_tablet_manager.cc:1428] T d6e12785b29a4efc823770a057a8de7c P 6a3774b03adf44d3b1475982c04da6aa: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 18:33:39.061278 19386 raft_consensus.cc:374] T d6e12785b29a4efc823770a057a8de7c P f25e947f8c2c4fef895cfd78048211d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:33:39.061971 19386 raft_consensus.cc:729] T d6e12785b29a4efc823770a057a8de7c P f25e947f8c2c4fef895cfd78048211d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f25e947f8c2c4fef895cfd78048211d7, State: Initialized, Role: FOLLOWER
I20260807 18:33:39.062201 19386 consensus_queue.cc:260] T d6e12785b29a4efc823770a057a8de7c P f25e947f8c2c4fef895cfd78048211d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f25e947f8c2c4fef895cfd78048211d7" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 39193 } } peers { permanent_uuid: "6a3774b03adf44d3b1475982c04da6aa" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 42159 } } peers { permanent_uuid: "a290fe9b958d48f0a540e728e58c4bc7" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 33879 } }
I20260807 18:33:39.062623 19386 ts_tablet_manager.cc:1428] T d6e12785b29a4efc823770a057a8de7c P f25e947f8c2c4fef895cfd78048211d7: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260807 18:33:39.063994 18991 heartbeater.cc:503] Master 127.11.14.124:36503 was elected leader, sending a full tablet report...
I20260807 18:33:39.065193 19397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:39.063862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:45409 (local address 127.11.14.66:40027)
0807 18:33:39.064960 (+ 1098us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:39.065034 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.126:45409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1021,"thread_start_us":117,"threads_started":1}
I20260807 18:33:39.065856 19286 heartbeater.cc:503] Master 127.11.14.124:36503 was elected leader, sending a full tablet report...
I20260807 18:33:39.065914 19394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:39.061699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:45409 (local address 127.11.14.65:39803)
0807 18:33:39.065727 (+ 4028us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:39.065799 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.126:45409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":3939,"thread_start_us":111,"threads_started":1}
I20260807 18:33:39.066843 19394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:39.066377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:45409 (local address 127.11.14.65:47499)
0807 18:33:39.066685 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:39.066753 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.126:45409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":102,"threads_started":1}
I20260807 18:33:39.067193 19138 heartbeater.cc:503] Master 127.11.14.124:36503 was elected leader, sending a full tablet report...
I20260807 18:33:39.067754 19397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:39.067428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:45409 (local address 127.11.14.66:44909)
0807 18:33:39.067618 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:39.067681 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.126:45409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99}
I20260807 18:33:39.068737 19396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:39.063093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:45409 (local address 127.11.14.67:36575)
0807 18:33:39.068569 (+ 5476us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:39.068635 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.126:45409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":4560,"thread_start_us":106,"threads_started":1}
I20260807 18:33:39.069439 19396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:39.069227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:45409 (local address 127.11.14.67:54389)
0807 18:33:39.069330 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:39.069383 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.126:45409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260807 18:33:39.218372 19141 tablet.cc:2369] T d6e12785b29a4efc823770a057a8de7c P f25e947f8c2c4fef895cfd78048211d7: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 18:33:39.218772 19290 tablet.cc:2369] T d6e12785b29a4efc823770a057a8de7c P 6a3774b03adf44d3b1475982c04da6aa: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 18:33:39.236438 18994 tablet.cc:2369] T d6e12785b29a4efc823770a057a8de7c P a290fe9b958d48f0a540e728e58c4bc7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:33:39.287088 19395 raft_consensus.cc:482] T d6e12785b29a4efc823770a057a8de7c P f25e947f8c2c4fef895cfd78048211d7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:33:39.287259 19395 raft_consensus.cc:504] T d6e12785b29a4efc823770a057a8de7c P f25e947f8c2c4fef895cfd78048211d7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f25e947f8c2c4fef895cfd78048211d7" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 39193 } } peers { permanent_uuid: "6a3774b03adf44d3b1475982c04da6aa" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 42159 } } peers { permanent_uuid: "a290fe9b958d48f0a540e728e58c4bc7" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 33879 } }
I20260807 18:33:39.288725 19395 leader_election.cc:288] T d6e12785b29a4efc823770a057a8de7c P f25e947f8c2c4fef895cfd78048211d7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6a3774b03adf44d3b1475982c04da6aa (127.11.14.67:42159), a290fe9b958d48f0a540e728e58c4bc7 (127.11.14.65:33879)
I20260807 18:33:39.292613 19397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:39.288862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.67:42159 (local address 127.11.14.66:49843)
0807 18:33:39.289055 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:39.289074 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:39.289201 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:39.289520 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:39.289524 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:39.289541 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:33:39.289682 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:39.289688 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:39.290506 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:39.290510 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:39.291310 (+ 800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:39.291320 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:39.292159 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:39.292162 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:39.292283 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:39.292304 (+ 21us) client_negotiation.cc:769] Sending connection context
0807 18:33:39.292397 (+ 93us) client_negotiation.cc:240] Negotiation successful
0807 18:33:39.292472 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":71}
I20260807 18:33:39.292613 19384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:39.288936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.66:49843 (local address 127.11.14.67:42159)
0807 18:33:39.289102 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:39.289110 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:39.289158 (+ 48us) server_negotiation.cc:407] Connection header received
0807 18:33:39.289360 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:39.289365 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:39.289423 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:39.289512 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:33:39.289805 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:39.290373 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:39.291444 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:39.292011 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:39.292147 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:39.292205 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:39.292417 (+ 212us) server_negotiation.cc:299] Negotiation successful
0807 18:33:39.292489 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260807 18:33:39.293056 19235 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d6e12785b29a4efc823770a057a8de7c" candidate_uuid: "f25e947f8c2c4fef895cfd78048211d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a3774b03adf44d3b1475982c04da6aa" is_pre_election: true
I20260807 18:33:39.293219 19383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:39.289299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.66:58853 (local address 127.11.14.65:33879)
0807 18:33:39.289457 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:39.289463 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:39.289637 (+ 174us) server_negotiation.cc:407] Connection header received
0807 18:33:39.289764 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:39.289768 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:39.289821 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:39.289931 (+ 110us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:33:39.290237 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:39.290789 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:39.291969 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:39.292473 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:39.292614 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:39.292678 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:39.293009 (+ 331us) server_negotiation.cc:299] Negotiation successful
0807 18:33:39.293095 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260807 18:33:39.293228 19235 raft_consensus.cc:2454] T d6e12785b29a4efc823770a057a8de7c P 6a3774b03adf44d3b1475982c04da6aa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f25e947f8c2c4fef895cfd78048211d7 in term 0.
I20260807 18:33:39.293344 19399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:39.289177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.65:33879 (local address 127.11.14.66:58853)
0807 18:33:39.289527 (+ 350us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:39.289540 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:39.289662 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:39.289962 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:39.289965 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:39.289982 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:33:39.290103 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:39.290109 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:39.290913 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:39.290920 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:39.291835 (+ 915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:39.291845 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:39.292606 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:39.292610 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:39.292868 (+ 258us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:39.292893 (+ 25us) client_negotiation.cc:769] Sending connection context
0807 18:33:39.292972 (+ 79us) client_negotiation.cc:240] Negotiation successful
0807 18:33:39.293045 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":275,"thread_start_us":163,"threads_started":1}
I20260807 18:33:39.293676 19017 leader_election.cc:302] T d6e12785b29a4efc823770a057a8de7c P f25e947f8c2c4fef895cfd78048211d7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a3774b03adf44d3b1475982c04da6aa, f25e947f8c2c4fef895cfd78048211d7; no voters:
I20260807 18:33:39.293926 18940 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d6e12785b29a4efc823770a057a8de7c" candidate_uuid: "f25e947f8c2c4fef895cfd78048211d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a290fe9b958d48f0a540e728e58c4bc7" is_pre_election: true
I20260807 18:33:39.293989 19395 raft_consensus.cc:2781] T d6e12785b29a4efc823770a057a8de7c P f25e947f8c2c4fef895cfd78048211d7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:33:39.294080 19395 raft_consensus.cc:482] T d6e12785b29a4efc823770a057a8de7c P f25e947f8c2c4fef895cfd78048211d7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:33:39.294106 19395 raft_consensus.cc:3037] T d6e12785b29a4efc823770a057a8de7c P f25e947f8c2c4fef895cfd78048211d7 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:33:39.294232 18940 raft_consensus.cc:2454] T d6e12785b29a4efc823770a057a8de7c P a290fe9b958d48f0a540e728e58c4bc7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f25e947f8c2c4fef895cfd78048211d7 in term 0.
I20260807 18:33:39.295329 19395 raft_consensus.cc:504] T d6e12785b29a4efc823770a057a8de7c P f25e947f8c2c4fef895cfd78048211d7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f25e947f8c2c4fef895cfd78048211d7" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 39193 } } peers { permanent_uuid: "6a3774b03adf44d3b1475982c04da6aa" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 42159 } } peers { permanent_uuid: "a290fe9b958d48f0a540e728e58c4bc7" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 33879 } }
I20260807 18:33:39.295791 19395 leader_election.cc:288] T d6e12785b29a4efc823770a057a8de7c P f25e947f8c2c4fef895cfd78048211d7 [CANDIDATE]: Term 1 election: Requested vote from peers 6a3774b03adf44d3b1475982c04da6aa (127.11.14.67:42159), a290fe9b958d48f0a540e728e58c4bc7 (127.11.14.65:33879)
I20260807 18:33:39.296223 19235 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d6e12785b29a4efc823770a057a8de7c" candidate_uuid: "f25e947f8c2c4fef895cfd78048211d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a3774b03adf44d3b1475982c04da6aa"
I20260807 18:33:39.296309 18940 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d6e12785b29a4efc823770a057a8de7c" candidate_uuid: "f25e947f8c2c4fef895cfd78048211d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a290fe9b958d48f0a540e728e58c4bc7"
I20260807 18:33:39.296363 19235 raft_consensus.cc:3037] T d6e12785b29a4efc823770a057a8de7c P 6a3774b03adf44d3b1475982c04da6aa [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:33:39.296458 18940 raft_consensus.cc:3037] T d6e12785b29a4efc823770a057a8de7c P a290fe9b958d48f0a540e728e58c4bc7 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:33:39.297441 18940 raft_consensus.cc:2454] T d6e12785b29a4efc823770a057a8de7c P a290fe9b958d48f0a540e728e58c4bc7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f25e947f8c2c4fef895cfd78048211d7 in term 1.
I20260807 18:33:39.297490 19235 raft_consensus.cc:2454] T d6e12785b29a4efc823770a057a8de7c P 6a3774b03adf44d3b1475982c04da6aa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f25e947f8c2c4fef895cfd78048211d7 in term 1.
I20260807 18:33:39.297765 19017 leader_election.cc:302] T d6e12785b29a4efc823770a057a8de7c P f25e947f8c2c4fef895cfd78048211d7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a290fe9b958d48f0a540e728e58c4bc7, f25e947f8c2c4fef895cfd78048211d7; no voters:
I20260807 18:33:39.297991 19395 raft_consensus.cc:2781] T d6e12785b29a4efc823770a057a8de7c P f25e947f8c2c4fef895cfd78048211d7 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:33:39.298276 19395 raft_consensus.cc:686] T d6e12785b29a4efc823770a057a8de7c P f25e947f8c2c4fef895cfd78048211d7 [term 1 LEADER]: Becoming Leader. State: Replica: f25e947f8c2c4fef895cfd78048211d7, State: Running, Role: LEADER
I20260807 18:33:39.298595 19395 consensus_queue.cc:237] T d6e12785b29a4efc823770a057a8de7c P f25e947f8c2c4fef895cfd78048211d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f25e947f8c2c4fef895cfd78048211d7" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 39193 } } peers { permanent_uuid: "6a3774b03adf44d3b1475982c04da6aa" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 42159 } } peers { permanent_uuid: "a290fe9b958d48f0a540e728e58c4bc7" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 33879 } }
I20260807 18:33:39.299031 19399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:39.298796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:45409 (local address 127.11.14.66:57667)
0807 18:33:39.298905 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:39.298966 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.126:45409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260807 18:33:39.301590 18794 catalog_manager.cc:5495] T d6e12785b29a4efc823770a057a8de7c P f25e947f8c2c4fef895cfd78048211d7 reported cstate change: term changed from 0 to 1, leader changed from <none> to f25e947f8c2c4fef895cfd78048211d7 (127.11.14.66). New cstate: current_term: 1 leader_uuid: "f25e947f8c2c4fef895cfd78048211d7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f25e947f8c2c4fef895cfd78048211d7" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 39193 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6a3774b03adf44d3b1475982c04da6aa" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 42159 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a290fe9b958d48f0a540e728e58c4bc7" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 33879 } health_report { overall_health: UNKNOWN } } }
I20260807 18:33:39.347728 19380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:39.344443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35420 (local address 127.11.14.66:39193)
0807 18:33:39.344573 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:39.344577 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:39.344591 (+ 14us) server_negotiation.cc:407] Connection header received
0807 18:33:39.344673 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:39.344675 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:39.344752 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:39.344848 (+ 96us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:33:39.345065 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:39.345571 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:39.346356 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:39.346566 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:39.346643 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:39.346891 (+ 248us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:33:39.347348 (+ 457us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:33:39.347436 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:39.347500 (+ 64us) server_negotiation.cc:299] Negotiation successful
0807 18:33:39.347550 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260807 18:33:39.350684 19235 raft_consensus.cc:1261] T d6e12785b29a4efc823770a057a8de7c P 6a3774b03adf44d3b1475982c04da6aa [term 1 FOLLOWER]: Refusing update from remote peer f25e947f8c2c4fef895cfd78048211d7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:33:39.350865 18940 raft_consensus.cc:1261] T d6e12785b29a4efc823770a057a8de7c P a290fe9b958d48f0a540e728e58c4bc7 [term 1 FOLLOWER]: Refusing update from remote peer f25e947f8c2c4fef895cfd78048211d7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:33:39.351275 19400 consensus_queue.cc:1035] T d6e12785b29a4efc823770a057a8de7c P f25e947f8c2c4fef895cfd78048211d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a3774b03adf44d3b1475982c04da6aa" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 42159 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:33:39.351516 19395 consensus_queue.cc:1035] T d6e12785b29a4efc823770a057a8de7c P f25e947f8c2c4fef895cfd78048211d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a290fe9b958d48f0a540e728e58c4bc7" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 33879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:33:39.353211 19396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:39.352921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:45409 (local address 127.11.14.67:59495)
0807 18:33:39.353080 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:39.353140 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.126:45409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91}
I20260807 18:33:39.354230 19398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:39.353817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:45409 (local address 127.11.14.65:48595)
0807 18:33:39.354074 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:39.354137 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.126:45409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":189}
I20260807 18:33:39.357609 19407 mvcc.cc:205] Tried to move back new op lower bound from 7315978728855060480 to 7315978728650334208. Current Snapshot: MvccSnapshot[applied={T|T < 7315978728855060480}]
I20260807 18:33:39.358853 19398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:39.357485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:45409 (local address 127.11.14.65:58267)
0807 18:33:39.358728 (+ 1243us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:39.358785 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.126:45409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1169}
I20260807 18:33:39.359161 19399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:39.358771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:45409 (local address 127.11.14.66:54023)
0807 18:33:39.359037 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:39.359093 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.126:45409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":201}
I20260807 18:33:39.361711 19396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:39.361423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:45409 (local address 127.11.14.67:38967)
0807 18:33:39.361537 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:39.361594 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.126:45409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260807 18:33:39.364666 18796 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44620:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:33:39.364835 18796 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44620:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:33:39.368741 18796 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ab43b0280a964c18a81ab0531a0a2a10: Sending DeleteTablet for 3 replicas of tablet d6e12785b29a4efc823770a057a8de7c
I20260807 18:33:39.369902 19214 tablet_service.cc:1514] Processing DeleteTablet for tablet d6e12785b29a4efc823770a057a8de7c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:33:39 UTC) from {username='oryx', principal='oryx/127.11.14.124@KRBTEST.COM'} at 127.0.0.1:36510
I20260807 18:33:39.370127 11321 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkz56XN/build/debug/bin/kudu with pid 18752
I20260807 18:33:39.370303 19068 tablet_service.cc:1514] Processing DeleteTablet for tablet d6e12785b29a4efc823770a057a8de7c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:33:39 UTC) from {username='oryx', principal='oryx/127.11.14.124@KRBTEST.COM'} at 127.0.0.1:35412
I20260807 18:33:39.371183 19417 tablet_replica.cc:330] T d6e12785b29a4efc823770a057a8de7c P f25e947f8c2c4fef895cfd78048211d7: stopping tablet replica
I20260807 18:33:39.371326 19417 raft_consensus.cc:2229] T d6e12785b29a4efc823770a057a8de7c P f25e947f8c2c4fef895cfd78048211d7 [term 1 LEADER]: Raft consensus shutting down.
I20260807 18:33:39.371670 19417 raft_consensus.cc:2258] T d6e12785b29a4efc823770a057a8de7c P f25e947f8c2c4fef895cfd78048211d7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:33:39.371891 19416 tablet_replica.cc:330] T d6e12785b29a4efc823770a057a8de7c P 6a3774b03adf44d3b1475982c04da6aa: stopping tablet replica
I20260807 18:33:39.372005 19416 raft_consensus.cc:2229] T d6e12785b29a4efc823770a057a8de7c P 6a3774b03adf44d3b1475982c04da6aa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:33:39.372136 19416 raft_consensus.cc:2258] T d6e12785b29a4efc823770a057a8de7c P 6a3774b03adf44d3b1475982c04da6aa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:33:39.373121 19417 ts_tablet_manager.cc:1905] T d6e12785b29a4efc823770a057a8de7c P f25e947f8c2c4fef895cfd78048211d7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:33:39.373250 19416 ts_tablet_manager.cc:1905] T d6e12785b29a4efc823770a057a8de7c P 6a3774b03adf44d3b1475982c04da6aa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:33:39.375880 19416 ts_tablet_manager.cc:1918] T d6e12785b29a4efc823770a057a8de7c P 6a3774b03adf44d3b1475982c04da6aa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:33:39.375880 19417 ts_tablet_manager.cc:1918] T d6e12785b29a4efc823770a057a8de7c P f25e947f8c2c4fef895cfd78048211d7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:33:39.375972 19416 log.cc:1196] T d6e12785b29a4efc823770a057a8de7c P 6a3774b03adf44d3b1475982c04da6aa: Deleting WAL directory at /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/minicluster-data/ts-2/wal/wals/d6e12785b29a4efc823770a057a8de7c
I20260807 18:33:39.376096 19417 log.cc:1196] T d6e12785b29a4efc823770a057a8de7c P f25e947f8c2c4fef895cfd78048211d7: Deleting WAL directory at /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786127579900523-11321-0/minicluster-data/ts-1/wal/wals/d6e12785b29a4efc823770a057a8de7c
I20260807 18:33:39.376307 19416 ts_tablet_manager.cc:1939] T d6e12785b29a4efc823770a057a8de7c P 6a3774b03adf44d3b1475982c04da6aa: Deleting consensus metadata
I20260807 18:33:39.376529 19417 ts_tablet_manager.cc:1939] T d6e12785b29a4efc823770a057a8de7c P f25e947f8c2c4fef895cfd78048211d7: Deleting consensus metadata
I20260807 18:33:39.377120 18780 catalog_manager.cc:4841] TS 6a3774b03adf44d3b1475982c04da6aa (127.11.14.67:42159): tablet d6e12785b29a4efc823770a057a8de7c (table test-table [id=10fa4c5ee15d4d1f8e3fa3ed78a46dd7]) successfully deleted
I20260807 18:33:39.377200 18781 catalog_manager.cc:4841] TS f25e947f8c2c4fef895cfd78048211d7 (127.11.14.66:39193): tablet d6e12785b29a4efc823770a057a8de7c (table test-table [id=10fa4c5ee15d4d1f8e3fa3ed78a46dd7]) successfully deleted
W20260807 18:33:39.391351 18780 connection.cc:535] client connection to 127.11.14.65:33879 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260807 18:33:39.392138 11321 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkz56XN/build/debug/bin/kudu with pid 18999
I20260807 18:33:39.392490 19382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:39.392117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.65:33879 (local address 127.0.0.1:60386)
0807 18:33:39.392296 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:39.392377 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.65:33879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260807 18:33:39.393266 18780 catalog_manager.cc:4584] TS a290fe9b958d48f0a540e728e58c4bc7 (127.11.14.65:33879): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d6e12785b29a4efc823770a057a8de7c: Network error: Client connection negotiation failed: client connection to 127.11.14.65:33879: connect: Connection refused (error 111)
I20260807 18:33:39.401036 11321 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkz56XN/build/debug/bin/kudu with pid 19146
I20260807 18:33:39.409219 11321 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkz56XN/build/debug/bin/kudu with pid 18451
I20260807 18:33:39.417958 11321 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkz56XN/build/debug/bin/kudu with pid 18571
2026-08-07T18:33:39Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (5761 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127579900523-11321-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:33:39 dist-test-slave-6k22 krb5kdc[19420](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:33:39 dist-test-slave-6k22 krb5kdc[19420](info): set up 2 sockets
Aug 07 18:33:39 dist-test-slave-6k22 krb5kdc[19420](info): commencing operation
krb5kdc: starting...
W20260807 18:33:40.626168 11321 mini_kdc.cc:121] Time spent starting KDC: real 1.182s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127579900523-11321-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127579900523-11321-0/krb5kdc/test-user.keytab.
Aug 07 18:33:40 dist-test-slave-6k22 krb5kdc[19420](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127620, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:33:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:33:40Z Disabled control of system clock
WARNING: no policy specified for oryx/127.11.14.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.14.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.14.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127579900523-11321-0/krb5kdc/oryx_127.11.14.126.keytab.
Entry for principal oryx/127.11.14.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127579900523-11321-0/krb5kdc/oryx_127.11.14.126.keytab.
I20260807 18:33:40.756184 11321 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkz56XN/build/debug/bin/kudu
/tmp/dist-test-taskkz56XN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127579900523-11321-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127579900523-11321-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127579900523-11321-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127579900523-11321-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.14.126:45977
--webserver_interface=127.11.14.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127579900523-11321-0/krb5kdc/oryx_127.11.14.126.keytab
--principal=oryx/127.11.14.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.14.84:35447
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.14.126:45977
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127579900523-11321-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127579900523-11321-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127579900523-11321-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:33:40.871443 19433 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:33:40.871793 19433 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:33:40.871874 19433 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:33:40.875922 19433 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:33:40.875993 19433 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260807 18:33:40.876026 19433 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:33:40.876088 19433 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:33:40.876142 19433 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:33:40.876178 19433 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:33:40.881488 19433 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.14.84:35447
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127579900523-11321-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127579900523-11321-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.14.126:45977
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127579900523-11321-0/krb5kdc/oryx_127.11.14.126.keytab
--principal=oryx/127.11.14.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.14.126:45977
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127579900523-11321-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.14.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19433
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127579900523-11321-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:33:40.882534 19433 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:33:40.883538 19433 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:33:40.892182 19439 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:33:40.892311 19441 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:33:40.892194 19433 server_base.cc:1034] running on GCE node
W20260807 18:33:40.892194 19438 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:33:40.893050 19433 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:33:40.894147 19433 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:33:40.895346 19433 hybrid_clock.cc:648] HybridClock initialized: now 1786127620895332 us; error 25 us; skew 500 ppm
Aug 07 18:33:40 dist-test-slave-6k22 krb5kdc[19420](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127620, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.14.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:33:40.898087 19433 init.cc:377] Logged in from keytab as oryx/127.11.14.126@KRBTEST.COM (short username oryx)
I20260807 18:33:40.898989 19433 webserver.cc:460] Webserver started at http://127.11.14.126:45813/ using document root <none> and password file <none>
I20260807 18:33:40.899513 19433 fs_manager.cc:359] Metadata directory not provided
I20260807 18:33:40.899632 19433 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:33:40.899827 19433 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:33:40.901827 19433 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127579900523-11321-0/minicluster-data/master-0/data/instance:
uuid: "8482b9c6ca944b908d05b969396896b8"
format_stamp: "Formatted at 2026-08-07 18:33:40 on dist-test-slave-6k22"
server_key: "74595200a17dc2d84c39403ce798cb8f"
server_key_iv: "566ab10871d1bc7ce09a70fe99beaf15"
server_key_version: "encryptionkey@0"
I20260807 18:33:40.902420 19433 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127579900523-11321-0/minicluster-data/master-0/wal/instance:
uuid: "8482b9c6ca944b908d05b969396896b8"
format_stamp: "Formatted at 2026-08-07 18:33:40 on dist-test-slave-6k22"
server_key: "74595200a17dc2d84c39403ce798cb8f"
server_key_iv: "566ab10871d1bc7ce09a70fe99beaf15"
server_key_version: "encryptionkey@0"
I20260807 18:33:40.906114 19433 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.006s
I20260807 18:33:40.908594 19448 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:33:40.909816 19433 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 18:33:40.909943 19433 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127579900523-11321-0/minicluster-data/master-0/data,/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127579900523-11321-0/minicluster-data/master-0/wal
uuid: "8482b9c6ca944b908d05b969396896b8"
format_stamp: "Formatted at 2026-08-07 18:33:40 on dist-test-slave-6k22"
server_key: "74595200a17dc2d84c39403ce798cb8f"
server_key_iv: "566ab10871d1bc7ce09a70fe99beaf15"
server_key_version: "encryptionkey@0"
I20260807 18:33:40.910045 19433 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127579900523-11321-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127579900523-11321-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127579900523-11321-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:33:40.928447 19433 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:33:40.931139 19433 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:33:40.931360 19433 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:33:40.939375 19433 rpc_server.cc:307] RPC server started. Bound to: 127.11.14.126:45977
I20260807 18:33:40.939395 19513 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.14.126:45977 every 8 connection(s)
I20260807 18:33:40.940356 19433 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127579900523-11321-0/minicluster-data/master-0/data/info.pb
I20260807 18:33:40.942999 11321 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkz56XN/build/debug/bin/kudu as pid 19433
I20260807 18:33:40.943127 11321 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127579900523-11321-0/minicluster-data/master-0/wal/instance
I20260807 18:33:40.943473 11321 external_mini_cluster.cc:1428] Setting key 5e73782a8b57e8f266136a16cdb2e1a5
I20260807 18:33:40.943500 19514 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:33:40.951195 19514 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8482b9c6ca944b908d05b969396896b8: Bootstrap starting.
Aug 07 18:33:40 dist-test-slave-6k22 krb5kdc[19420](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127620, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.11.14.126@KRBTEST.COM
I20260807 18:33:40.953881 19514 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8482b9c6ca944b908d05b969396896b8: Neither blocks nor log segments found. Creating new log.
I20260807 18:33:40.955329 19514 log.cc:824] T 00000000000000000000000000000000 P 8482b9c6ca944b908d05b969396896b8: Log is configured to *not* fsync() on all Append() calls
I20260807 18:33:40.957751 19514 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8482b9c6ca944b908d05b969396896b8: No bootstrap required, opened a new log
I20260807 18:33:40.959139 19517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:40.945038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52774 (local address 127.11.14.126:45977)
0807 18:33:40.945472 (+ 434us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:40.945493 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:40.945536 (+ 43us) server_negotiation.cc:407] Connection header received
0807 18:33:40.946394 (+ 858us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:40.946421 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:40.946872 (+ 451us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:40.947294 (+ 422us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:33:40.948084 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:40.949586 (+ 1502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:40.950473 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:40.951182 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:40.951364 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:40.953964 (+ 2600us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:33:40.953970 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:33:40.953988 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:33:40.954024 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:33:40.956026 (+ 2002us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:40.956500 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:40.956507 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:40.956513 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:40.956650 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:40.956916 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:40.956921 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:40.956924 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:40.957466 (+ 542us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:33:40.957668 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:40.957877 (+ 209us) server_negotiation.cc:299] Negotiation successful
0807 18:33:40.958185 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":280,"thread_start_us":107,"threads_started":1}
I20260807 18:33:40.961530 19514 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8482b9c6ca944b908d05b969396896b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8482b9c6ca944b908d05b969396896b8" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 45977 } }
I20260807 18:33:40.961751 19514 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8482b9c6ca944b908d05b969396896b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:33:40.961788 19514 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8482b9c6ca944b908d05b969396896b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8482b9c6ca944b908d05b969396896b8, State: Initialized, Role: FOLLOWER
I20260807 18:33:40.962155 19514 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8482b9c6ca944b908d05b969396896b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8482b9c6ca944b908d05b969396896b8" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 45977 } }
I20260807 18:33:40.962266 19514 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8482b9c6ca944b908d05b969396896b8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:33:40.962311 19514 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8482b9c6ca944b908d05b969396896b8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:33:40.962385 19514 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8482b9c6ca944b908d05b969396896b8 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:33:40.963261 19514 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8482b9c6ca944b908d05b969396896b8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8482b9c6ca944b908d05b969396896b8" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 45977 } }
I20260807 18:33:40.963605 19514 leader_election.cc:302] T 00000000000000000000000000000000 P 8482b9c6ca944b908d05b969396896b8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8482b9c6ca944b908d05b969396896b8; no voters:
I20260807 18:33:40.963897 19514 leader_election.cc:288] T 00000000000000000000000000000000 P 8482b9c6ca944b908d05b969396896b8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:33:40.964017 19519 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8482b9c6ca944b908d05b969396896b8 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:33:40.964282 19519 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8482b9c6ca944b908d05b969396896b8 [term 1 LEADER]: Becoming Leader. State: Replica: 8482b9c6ca944b908d05b969396896b8, State: Running, Role: LEADER
I20260807 18:33:40.964679 19519 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8482b9c6ca944b908d05b969396896b8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8482b9c6ca944b908d05b969396896b8" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 45977 } }
I20260807 18:33:40.965173 19514 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8482b9c6ca944b908d05b969396896b8 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:33:40.966444 19521 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8482b9c6ca944b908d05b969396896b8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8482b9c6ca944b908d05b969396896b8. Latest consensus state: current_term: 1 leader_uuid: "8482b9c6ca944b908d05b969396896b8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8482b9c6ca944b908d05b969396896b8" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 45977 } } }
I20260807 18:33:40.966580 19521 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8482b9c6ca944b908d05b969396896b8 [sys.catalog]: This master's current role is: LEADER
I20260807 18:33:40.967037 19529 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:33:40.966998 19520 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8482b9c6ca944b908d05b969396896b8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8482b9c6ca944b908d05b969396896b8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8482b9c6ca944b908d05b969396896b8" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 45977 } } }
I20260807 18:33:40.967454 19520 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8482b9c6ca944b908d05b969396896b8 [sys.catalog]: This master's current role is: LEADER
I20260807 18:33:40.970297 19529 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:33:40.982283 19529 catalog_manager.cc:1347] Generated new cluster ID: b6ca1a2eaad24c03a9a2e036fad0f4aa
I20260807 18:33:40.982445 19529 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:33:40.998504 19529 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:33:40.999815 19529 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:33:41.012373 19529 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8482b9c6ca944b908d05b969396896b8: Generated new TSK 0
I20260807 18:33:41.013147 19529 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.11.14.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.14.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.14.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127579900523-11321-0/krb5kdc/oryx_127.11.14.65.keytab.
Entry for principal oryx/127.11.14.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127579900523-11321-0/krb5kdc/oryx_127.11.14.65.keytab.
I20260807 18:33:41.054052 11321 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkz56XN/build/debug/bin/kudu
/tmp/dist-test-taskkz56XN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127579900523-11321-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127579900523-11321-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127579900523-11321-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127579900523-11321-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.14.65:0
--local_ip_for_outbound_sockets=127.11.14.65
--webserver_interface=127.11.14.65
--webserver_port=0
--tserver_master_addrs=127.11.14.126:45977
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127579900523-11321-0/krb5kdc/oryx_127.11.14.65.keytab
--principal=oryx/127.11.14.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.14.84:35447
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127579900523-11321-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127579900523-11321-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127579900523-11321-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:33:41.191087 19541 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:33:41.191322 19541 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:33:41.191365 19541 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:33:41.195376 19541 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:33:41.195429 19541 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:33:41.195497 19541 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 18:33:41.195591 19541 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.14.65
I20260807 18:33:41.200822 19541 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.14.84:35447
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127579900523-11321-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127579900523-11321-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127579900523-11321-0/krb5kdc/oryx_127.11.14.65.keytab
--principal=oryx/127.11.14.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.14.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127579900523-11321-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.14.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.11.14.126:45977
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19541
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.14.65
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127579900523-11321-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:33:41.201809 19541 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:33:41.202790 19541 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:33:41.212718 19547 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:33:41.212783 19541 server_base.cc:1034] running on GCE node
W20260807 18:33:41.212684 19546 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:33:41.212692 19549 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:33:41.213363 19541 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:33:41.214027 19541 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:33:41.215207 19541 hybrid_clock.cc:648] HybridClock initialized: now 1786127621215185 us; error 40 us; skew 500 ppm
Aug 07 18:33:41 dist-test-slave-6k22 krb5kdc[19420](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127621, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.14.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:33:41.218003 19541 init.cc:377] Logged in from keytab as oryx/127.11.14.65@KRBTEST.COM (short username oryx)
I20260807 18:33:41.218855 19541 webserver.cc:460] Webserver started at http://127.11.14.65:32829/ using document root <none> and password file <none>
I20260807 18:33:41.219419 19541 fs_manager.cc:359] Metadata directory not provided
I20260807 18:33:41.219480 19541 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:33:41.219734 19541 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:33:41.221921 19541 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127579900523-11321-0/minicluster-data/ts-0/data/instance:
uuid: "b8b91f4589f343c88246ca60212c015a"
format_stamp: "Formatted at 2026-08-07 18:33:41 on dist-test-slave-6k22"
server_key: "2b1b19977be4a48e248bcc089f43fa9a"
server_key_iv: "d430d3c3c7d9e224d2f53bc8b72862a3"
server_key_version: "encryptionkey@0"
I20260807 18:33:41.222512 19541 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127579900523-11321-0/minicluster-data/ts-0/wal/instance:
uuid: "b8b91f4589f343c88246ca60212c015a"
format_stamp: "Formatted at 2026-08-07 18:33:41 on dist-test-slave-6k22"
server_key: "2b1b19977be4a48e248bcc089f43fa9a"
server_key_iv: "d430d3c3c7d9e224d2f53bc8b72862a3"
server_key_version: "encryptionkey@0"
I20260807 18:33:41.226287 19541 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260807 18:33:41.228843 19556 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:33:41.230110 19541 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260807 18:33:41.230253 19541 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127579900523-11321-0/minicluster-data/ts-0/data,/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127579900523-11321-0/minicluster-data/ts-0/wal
uuid: "b8b91f4589f343c88246ca60212c015a"
format_stamp: "Formatted at 2026-08-07 18:33:41 on dist-test-slave-6k22"
server_key: "2b1b19977be4a48e248bcc089f43fa9a"
server_key_iv: "d430d3c3c7d9e224d2f53bc8b72862a3"
server_key_version: "encryptionkey@0"
I20260807 18:33:41.230367 19541 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127579900523-11321-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127579900523-11321-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127579900523-11321-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:33:41.261927 19541 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:33:41.265168 19541 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:33:41.265380 19541 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:33:41.267156 19541 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:33:41.267230 19541 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:33:41.267303 19541 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:33:41.267364 19541 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 18:33:41 dist-test-slave-6k22 krb5kdc[19420](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127621, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.14.65@KRBTEST.COM for oryx/127.11.14.126@KRBTEST.COM
I20260807 18:33:41.279276 19541 rpc_server.cc:307] RPC server started. Bound to: 127.11.14.65:41543
I20260807 18:33:41.280603 19541 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127579900523-11321-0/minicluster-data/ts-0/data/info.pb
I20260807 18:33:41.281807 19677 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.14.65:41543 every 8 connection(s)
I20260807 18:33:41.282922 11321 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkz56XN/build/debug/bin/kudu as pid 19541
I20260807 18:33:41.283057 11321 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127579900523-11321-0/minicluster-data/ts-0/wal/instance
I20260807 18:33:41.283380 11321 external_mini_cluster.cc:1428] Setting key 013133bd51ce8ea40ea1e622b569d0b0
I20260807 18:33:41.284186 19567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:41.268367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:45977 (local address 127.11.14.65:55997)
0807 18:33:41.268840 (+ 473us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:41.268891 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:41.269883 (+ 992us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:41.270824 (+ 941us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:41.270835 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:41.271307 (+ 472us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:33:41.271913 (+ 606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:41.271929 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:41.272994 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:41.272997 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:41.273973 (+ 976us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:41.273985 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:41.274754 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:41.274758 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:41.274993 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:41.275747 (+ 754us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:33:41.275772 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:33:41.278003 (+ 2231us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:33:41.280659 (+ 2656us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:33:41.280664 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:33:41.280681 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:33:41.281092 (+ 411us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:33:41.281442 (+ 350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:33:41.281448 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:33:41.281451 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:33:41.281634 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:33:41.282154 (+ 520us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:33:41.282161 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:33:41.282492 (+ 331us) client_negotiation.cc:769] Sending connection context
0807 18:33:41.282738 (+ 246us) client_negotiation.cc:240] Negotiation successful
0807 18:33:41.283137 (+ 399us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":131,"threads_started":1}
I20260807 18:33:41.284255 19517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:41.267949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.65:55997 (local address 127.11.14.126:45977)
0807 18:33:41.268119 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:41.268124 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:41.269007 (+ 883us) server_negotiation.cc:407] Connection header received
0807 18:33:41.270149 (+ 1142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:41.270153 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:41.270218 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:41.270284 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:33:41.272064 (+ 1780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:41.272860 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:41.274116 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:41.274428 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:41.274537 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:41.278447 (+ 3910us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:33:41.278453 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:33:41.278459 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:33:41.278489 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:33:41.280480 (+ 1991us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:41.281223 (+ 743us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:41.281226 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:41.281228 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:41.281330 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:41.281730 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:41.281734 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:41.281735 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:41.281995 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:33:41.282099 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:41.283965 (+ 1866us) server_negotiation.cc:299] Negotiation successful
0807 18:33:41.284098 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260807 18:33:41.285715 19678 heartbeater.cc:347] Connected to a master server at 127.11.14.126:45977
I20260807 18:33:41.285861 19678 heartbeater.cc:464] Registering TS with master...
I20260807 18:33:41.286401 19678 heartbeater.cc:511] Master 127.11.14.126:45977 requested a full tablet report, sending...
I20260807 18:33:41.288142 19464 ts_manager.cc:194] Registered new tserver with Master: b8b91f4589f343c88246ca60212c015a (127.11.14.65:41543)
I20260807 18:33:41.289342 19464 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.11.14.65@KRBTEST.COM'} at 127.11.14.65:55997
WARNING: no policy specified for oryx/127.11.14.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.14.66@KRBTEST.COM" created.
I20260807 18:33:41.299774 19517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:41.291987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.65:40291 (local address 127.11.14.126:45977)
0807 18:33:41.292135 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:41.292141 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:41.292235 (+ 94us) server_negotiation.cc:407] Connection header received
0807 18:33:41.292316 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:41.292321 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:41.292376 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:41.292474 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:33:41.292878 (+ 404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:41.293763 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:41.294654 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:41.295038 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:41.295165 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:41.296284 (+ 1119us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:33:41.296287 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:33:41.296289 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:33:41.296311 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:33:41.297793 (+ 1482us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:41.298374 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:41.298377 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:41.298378 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:41.298455 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:41.298786 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:41.298789 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:41.298792 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:41.299080 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:33:41.299181 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:41.299492 (+ 311us) server_negotiation.cc:299] Negotiation successful
0807 18:33:41.299660 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260807 18:33:41.299899 19685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:41.291864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:45977 (local address 127.11.14.65:40291)
0807 18:33:41.292148 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:41.292161 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:41.292244 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:41.292495 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:41.292497 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:41.292663 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:33:41.292759 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:41.292764 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:41.293894 (+ 1130us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:41.293897 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:41.294528 (+ 631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:41.294538 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:41.295156 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:41.295158 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:41.295227 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:41.295763 (+ 536us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:33:41.295775 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:33:41.296141 (+ 366us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:33:41.297926 (+ 1785us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:33:41.297929 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:33:41.297930 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:33:41.298265 (+ 335us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:33:41.298540 (+ 275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:33:41.298543 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:33:41.298544 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:33:41.298628 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:33:41.299183 (+ 555us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:33:41.299185 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:33:41.299267 (+ 82us) client_negotiation.cc:769] Sending connection context
0807 18:33:41.299407 (+ 140us) client_negotiation.cc:240] Negotiation successful
0807 18:33:41.299589 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":118,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.14.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127579900523-11321-0/krb5kdc/oryx_127.11.14.66.keytab.
Entry for principal oryx/127.11.14.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127579900523-11321-0/krb5kdc/oryx_127.11.14.66.keytab.
I20260807 18:33:41.314636 11321 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkz56XN/build/debug/bin/kudu
/tmp/dist-test-taskkz56XN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127579900523-11321-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127579900523-11321-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127579900523-11321-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127579900523-11321-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.14.66:0
--local_ip_for_outbound_sockets=127.11.14.66
--webserver_interface=127.11.14.66
--webserver_port=0
--tserver_master_addrs=127.11.14.126:45977
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127579900523-11321-0/krb5kdc/oryx_127.11.14.66.keytab
--principal=oryx/127.11.14.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.14.84:35447
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127579900523-11321-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127579900523-11321-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127579900523-11321-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:33:41.429651 19687 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:33:41.429929 19687 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:33:41.430029 19687 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:33:41.434177 19687 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:33:41.434271 19687 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:33:41.434347 19687 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 18:33:41.434440 19687 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.14.66
I20260807 18:33:41.439805 19687 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.14.84:35447
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127579900523-11321-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127579900523-11321-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127579900523-11321-0/krb5kdc/oryx_127.11.14.66.keytab
--principal=oryx/127.11.14.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.14.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127579900523-11321-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.14.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.11.14.126:45977
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19687
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.14.66
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127579900523-11321-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:33:41.440933 19687 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:33:41.441989 19687 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 18:33:41.452205 19687 server_base.cc:1034] running on GCE node
W20260807 18:33:41.452164 19693 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:33:41.452160 19692 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:33:41.452149 19695 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:33:41.452744 19687 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:33:41.453465 19687 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:33:41.454686 19687 hybrid_clock.cc:648] HybridClock initialized: now 1786127621454679 us; error 29 us; skew 500 ppm
Aug 07 18:33:41 dist-test-slave-6k22 krb5kdc[19420](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127621, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.14.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:33:41.457578 19687 init.cc:377] Logged in from keytab as oryx/127.11.14.66@KRBTEST.COM (short username oryx)
I20260807 18:33:41.458647 19687 webserver.cc:460] Webserver started at http://127.11.14.66:44075/ using document root <none> and password file <none>
I20260807 18:33:41.459221 19687 fs_manager.cc:359] Metadata directory not provided
I20260807 18:33:41.459318 19687 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:33:41.459605 19687 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:33:41.461731 19687 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127579900523-11321-0/minicluster-data/ts-1/data/instance:
uuid: "973fa32d73684f51949933c03a32ce8e"
format_stamp: "Formatted at 2026-08-07 18:33:41 on dist-test-slave-6k22"
server_key: "183d863904e36cdbbb77ccc44aae89bf"
server_key_iv: "acd4b704a1b4dca6deb2700ebedc45e3"
server_key_version: "encryptionkey@0"
I20260807 18:33:41.462319 19687 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127579900523-11321-0/minicluster-data/ts-1/wal/instance:
uuid: "973fa32d73684f51949933c03a32ce8e"
format_stamp: "Formatted at 2026-08-07 18:33:41 on dist-test-slave-6k22"
server_key: "183d863904e36cdbbb77ccc44aae89bf"
server_key_iv: "acd4b704a1b4dca6deb2700ebedc45e3"
server_key_version: "encryptionkey@0"
I20260807 18:33:41.466334 19687 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 18:33:41.468973 19702 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:33:41.470141 19687 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 18:33:41.470283 19687 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127579900523-11321-0/minicluster-data/ts-1/data,/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127579900523-11321-0/minicluster-data/ts-1/wal
uuid: "973fa32d73684f51949933c03a32ce8e"
format_stamp: "Formatted at 2026-08-07 18:33:41 on dist-test-slave-6k22"
server_key: "183d863904e36cdbbb77ccc44aae89bf"
server_key_iv: "acd4b704a1b4dca6deb2700ebedc45e3"
server_key_version: "encryptionkey@0"
I20260807 18:33:41.470396 19687 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127579900523-11321-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127579900523-11321-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127579900523-11321-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:33:41.482738 19687 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:33:41.486022 19687 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:33:41.486220 19687 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:33:41.487763 19687 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:33:41.487843 19687 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:33:41.487914 19687 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:33:41.487967 19687 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 18:33:41 dist-test-slave-6k22 krb5kdc[19420](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127621, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.14.66@KRBTEST.COM for oryx/127.11.14.126@KRBTEST.COM
I20260807 18:33:41.499536 19687 rpc_server.cc:307] RPC server started. Bound to: 127.11.14.66:43313
I20260807 18:33:41.500572 19687 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127579900523-11321-0/minicluster-data/ts-1/data/info.pb
I20260807 18:33:41.501300 11321 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkz56XN/build/debug/bin/kudu as pid 19687
I20260807 18:33:41.501458 11321 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127579900523-11321-0/minicluster-data/ts-1/wal/instance
I20260807 18:33:41.501745 11321 external_mini_cluster.cc:1428] Setting key 3217ac132ec946f1915de6ee6084a395
I20260807 18:33:41.502434 19517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:41.488455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.66:43067 (local address 127.11.14.126:45977)
0807 18:33:41.488611 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:41.488616 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:41.489618 (+ 1002us) server_negotiation.cc:407] Connection header received
0807 18:33:41.490978 (+ 1360us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:41.490981 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:41.491040 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:41.491138 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:33:41.492461 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:41.493177 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:41.494520 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:41.494742 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:41.494858 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:41.498420 (+ 3562us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:33:41.498424 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:33:41.498427 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:33:41.498460 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:33:41.500023 (+ 1563us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:41.500602 (+ 579us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:41.500605 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:41.500606 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:41.500720 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:41.501138 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:41.501141 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:41.501143 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:41.501408 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:33:41.501516 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:41.502134 (+ 618us) server_negotiation.cc:299] Negotiation successful
0807 18:33:41.502271 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260807 18:33:41.501353 19822 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.14.66:43313 every 8 connection(s)
I20260807 18:33:41.503269 19712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:41.488928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:45977 (local address 127.11.14.66:43067)
0807 18:33:41.489427 (+ 499us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:41.489487 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:41.490652 (+ 1165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:41.491447 (+ 795us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:41.491459 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:41.491844 (+ 385us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:33:41.492297 (+ 453us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:41.492310 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:41.493304 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:41.493308 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:41.494384 (+ 1076us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:41.494396 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:41.494895 (+ 499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:41.494900 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:41.495075 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:41.495736 (+ 661us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:33:41.495752 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:33:41.498196 (+ 2444us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:33:41.500138 (+ 1942us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:33:41.500144 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:33:41.500158 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:33:41.500502 (+ 344us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:33:41.500812 (+ 310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:33:41.500816 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:33:41.500818 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:33:41.501008 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:33:41.501572 (+ 564us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:33:41.501578 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:33:41.501849 (+ 271us) client_negotiation.cc:769] Sending connection context
0807 18:33:41.502090 (+ 241us) client_negotiation.cc:240] Negotiation successful
0807 18:33:41.502434 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":98,"threads_started":1}
I20260807 18:33:41.504558 19823 heartbeater.cc:347] Connected to a master server at 127.11.14.126:45977
I20260807 18:33:41.504696 19823 heartbeater.cc:464] Registering TS with master...
I20260807 18:33:41.505268 19823 heartbeater.cc:511] Master 127.11.14.126:45977 requested a full tablet report, sending...
I20260807 18:33:41.506451 19464 ts_manager.cc:194] Registered new tserver with Master: 973fa32d73684f51949933c03a32ce8e (127.11.14.66:43313)
I20260807 18:33:41.507277 19464 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.11.14.66@KRBTEST.COM'} at 127.11.14.66:43067
WARNING: no policy specified for oryx/127.11.14.67@KRBTEST.COM; defaulting to no policy
I20260807 18:33:41.515712 19517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:41.507872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.66:52499 (local address 127.11.14.126:45977)
0807 18:33:41.507993 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:41.507999 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:41.508240 (+ 241us) server_negotiation.cc:407] Connection header received
0807 18:33:41.508345 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:41.508351 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:41.508405 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:41.508491 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:33:41.509010 (+ 519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:41.509844 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:41.510652 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:41.511075 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:41.511187 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:41.512286 (+ 1099us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:33:41.512290 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:33:41.512293 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:33:41.512316 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:33:41.514061 (+ 1745us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:41.514503 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:41.514506 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:41.514508 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:41.514590 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:41.514890 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:41.514893 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:41.514894 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:41.515095 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:33:41.515181 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:41.515460 (+ 279us) server_negotiation.cc:299] Negotiation successful
0807 18:33:41.515552 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260807 18:33:41.515720 19831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:41.507811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:45977 (local address 127.11.14.66:52499)
0807 18:33:41.508144 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:41.508160 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:41.508259 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:41.508530 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:41.508533 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:41.508741 (+ 208us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:33:41.508869 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:41.508876 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:41.509983 (+ 1107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:41.509985 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:41.510524 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:41.510532 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:41.511200 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:41.511203 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:41.511269 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:41.511821 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:33:41.511833 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:33:41.512143 (+ 310us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:33:41.514176 (+ 2033us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:33:41.514179 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:33:41.514181 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:33:41.514386 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:33:41.514680 (+ 294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:33:41.514684 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:33:41.514685 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:33:41.514793 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:33:41.515187 (+ 394us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:33:41.515189 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:33:41.515265 (+ 76us) client_negotiation.cc:769] Sending connection context
0807 18:33:41.515392 (+ 127us) client_negotiation.cc:240] Negotiation successful
0807 18:33:41.515496 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":105,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.14.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.14.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127579900523-11321-0/krb5kdc/oryx_127.11.14.67.keytab.
Entry for principal oryx/127.11.14.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127579900523-11321-0/krb5kdc/oryx_127.11.14.67.keytab.
I20260807 18:33:41.532966 11321 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkz56XN/build/debug/bin/kudu
/tmp/dist-test-taskkz56XN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127579900523-11321-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127579900523-11321-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127579900523-11321-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127579900523-11321-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.14.67:0
--local_ip_for_outbound_sockets=127.11.14.67
--webserver_interface=127.11.14.67
--webserver_port=0
--tserver_master_addrs=127.11.14.126:45977
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127579900523-11321-0/krb5kdc/oryx_127.11.14.67.keytab
--principal=oryx/127.11.14.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.14.84:35447
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127579900523-11321-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127579900523-11321-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127579900523-11321-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:33:41.648427 19833 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:33:41.648665 19833 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:33:41.648710 19833 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:33:41.652755 19833 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:33:41.652817 19833 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:33:41.652855 19833 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 18:33:41.652889 19833 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.14.67
I20260807 18:33:41.658223 19833 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.14.84:35447
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127579900523-11321-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127579900523-11321-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127579900523-11321-0/krb5kdc/oryx_127.11.14.67.keytab
--principal=oryx/127.11.14.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.14.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127579900523-11321-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.14.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.11.14.126:45977
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19833
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.14.67
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127579900523-11321-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:33:41.659363 19833 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:33:41.660419 19833 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:33:41.670799 19838 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:33:41.670823 19839 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:33:41.671096 19833 server_base.cc:1034] running on GCE node
W20260807 18:33:41.671168 19841 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:33:41.671792 19833 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:33:41.672482 19833 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:33:41.673678 19833 hybrid_clock.cc:648] HybridClock initialized: now 1786127621673639 us; error 48 us; skew 500 ppm
Aug 07 18:33:41 dist-test-slave-6k22 krb5kdc[19420](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127621, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.14.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:33:41.676548 19833 init.cc:377] Logged in from keytab as oryx/127.11.14.67@KRBTEST.COM (short username oryx)
I20260807 18:33:41.677423 19833 webserver.cc:460] Webserver started at http://127.11.14.67:43643/ using document root <none> and password file <none>
I20260807 18:33:41.678004 19833 fs_manager.cc:359] Metadata directory not provided
I20260807 18:33:41.678097 19833 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:33:41.678354 19833 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:33:41.680495 19833 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127579900523-11321-0/minicluster-data/ts-2/data/instance:
uuid: "841d14d044a3450dba7a52ace1d8f9f9"
format_stamp: "Formatted at 2026-08-07 18:33:41 on dist-test-slave-6k22"
server_key: "5b21ded24683ddb89650ac0c5b3ba605"
server_key_iv: "ce8e7b77fe826ff3616fd59268277f9b"
server_key_version: "encryptionkey@0"
I20260807 18:33:41.681067 19833 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127579900523-11321-0/minicluster-data/ts-2/wal/instance:
uuid: "841d14d044a3450dba7a52ace1d8f9f9"
format_stamp: "Formatted at 2026-08-07 18:33:41 on dist-test-slave-6k22"
server_key: "5b21ded24683ddb89650ac0c5b3ba605"
server_key_iv: "ce8e7b77fe826ff3616fd59268277f9b"
server_key_version: "encryptionkey@0"
I20260807 18:33:41.685122 19833 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 18:33:41.687865 19849 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:33:41.689116 19833 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 18:33:41.689262 19833 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127579900523-11321-0/minicluster-data/ts-2/data,/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127579900523-11321-0/minicluster-data/ts-2/wal
uuid: "841d14d044a3450dba7a52ace1d8f9f9"
format_stamp: "Formatted at 2026-08-07 18:33:41 on dist-test-slave-6k22"
server_key: "5b21ded24683ddb89650ac0c5b3ba605"
server_key_iv: "ce8e7b77fe826ff3616fd59268277f9b"
server_key_version: "encryptionkey@0"
I20260807 18:33:41.689391 19833 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127579900523-11321-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127579900523-11321-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127579900523-11321-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:33:41.703680 19833 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:33:41.706573 19833 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:33:41.706722 19833 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:33:41.708201 19833 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:33:41.708254 19833 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:33:41.708295 19833 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:33:41.708359 19833 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 18:33:41 dist-test-slave-6k22 krb5kdc[19420](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127621, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.14.67@KRBTEST.COM for oryx/127.11.14.126@KRBTEST.COM
I20260807 18:33:41.720175 19833 rpc_server.cc:307] RPC server started. Bound to: 127.11.14.67:35051
I20260807 18:33:41.720191 19970 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.14.67:35051 every 8 connection(s)
I20260807 18:33:41.721277 19833 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127579900523-11321-0/minicluster-data/ts-2/data/info.pb
I20260807 18:33:41.722535 19517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:41.708888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.67:47207 (local address 127.11.14.126:45977)
0807 18:33:41.709013 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:41.709017 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:41.710085 (+ 1068us) server_negotiation.cc:407] Connection header received
0807 18:33:41.711040 (+ 955us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:41.711044 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:41.711106 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:41.711216 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:33:41.712641 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:41.713574 (+ 933us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:41.715031 (+ 1457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:41.715417 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:41.715549 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:41.718597 (+ 3048us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:33:41.718605 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:33:41.718611 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:33:41.718649 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:33:41.720355 (+ 1706us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:41.720848 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:41.720855 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:41.720859 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:41.720955 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:41.721288 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:41.721295 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:41.721297 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:41.721529 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:33:41.721647 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:41.722246 (+ 599us) server_negotiation.cc:299] Negotiation successful
0807 18:33:41.722372 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260807 18:33:41.723301 19859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:41.709430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:45977 (local address 127.11.14.67:47207)
0807 18:33:41.709934 (+ 504us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:41.709977 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:41.710823 (+ 846us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:41.711550 (+ 727us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:41.711561 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:41.711944 (+ 383us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:33:41.712486 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:41.712501 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:41.713747 (+ 1246us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:41.713751 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:41.714863 (+ 1112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:41.714874 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:41.715700 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:41.715707 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:41.715909 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:41.716555 (+ 646us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:33:41.716583 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:33:41.718409 (+ 1826us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:33:41.720470 (+ 2061us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:33:41.720475 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:33:41.720486 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:33:41.720732 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:33:41.721052 (+ 320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:33:41.721054 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:33:41.721055 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:33:41.721185 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:33:41.721661 (+ 476us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:33:41.721669 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:33:41.721957 (+ 288us) client_negotiation.cc:769] Sending connection context
0807 18:33:41.722172 (+ 215us) client_negotiation.cc:240] Negotiation successful
0807 18:33:41.722457 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":301,"thread_start_us":149,"threads_started":1}
I20260807 18:33:41.724624 19971 heartbeater.cc:347] Connected to a master server at 127.11.14.126:45977
I20260807 18:33:41.724802 19971 heartbeater.cc:464] Registering TS with master...
I20260807 18:33:41.725363 19971 heartbeater.cc:511] Master 127.11.14.126:45977 requested a full tablet report, sending...
I20260807 18:33:41.726397 19464 ts_manager.cc:194] Registered new tserver with Master: 841d14d044a3450dba7a52ace1d8f9f9 (127.11.14.67:35051)
I20260807 18:33:41.727192 19464 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.11.14.67@KRBTEST.COM'} at 127.11.14.67:47207
I20260807 18:33:41.730201 11321 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkz56XN/build/debug/bin/kudu as pid 19833
I20260807 18:33:41.730360 11321 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127579900523-11321-0/minicluster-data/ts-2/wal/instance
I20260807 18:33:41.730715 11321 external_mini_cluster.cc:1428] Setting key 710bf4f86ca9f792bc7a862671118c2f
I20260807 18:33:41.732999 11321 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 18:33:41.734476 19517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:41.726717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.67:55597 (local address 127.11.14.126:45977)
0807 18:33:41.726828 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:41.726832 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:41.727048 (+ 216us) server_negotiation.cc:407] Connection header received
0807 18:33:41.727149 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:41.727151 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:41.727186 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:41.727249 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:33:41.727794 (+ 545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:41.728553 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:41.729359 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:41.729563 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:41.729642 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:41.730797 (+ 1155us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:33:41.730803 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:33:41.730808 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:33:41.730841 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:33:41.732577 (+ 1736us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:41.733162 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:41.733166 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:41.733167 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:41.733251 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:41.733613 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:41.733615 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:41.733617 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:41.733782 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:33:41.733863 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:41.734268 (+ 405us) server_negotiation.cc:299] Negotiation successful
0807 18:33:41.734368 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260807 18:33:41.734637 19977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:41.726639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:45977 (local address 127.11.14.67:55597)
0807 18:33:41.726967 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:41.726981 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:41.727072 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:41.727298 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:41.727301 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:41.727521 (+ 220us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:33:41.727672 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:41.727680 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:41.728687 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:41.728690 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:41.729237 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:41.729245 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:41.729662 (+ 417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:41.729664 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:41.729732 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:41.730177 (+ 445us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:33:41.730195 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:33:41.730648 (+ 453us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:33:41.732675 (+ 2027us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:33:41.732677 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:33:41.732679 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:33:41.733005 (+ 326us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:33:41.733357 (+ 352us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:33:41.733361 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:33:41.733363 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:33:41.733462 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:33:41.733889 (+ 427us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:33:41.733895 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:33:41.734012 (+ 117us) client_negotiation.cc:769] Sending connection context
0807 18:33:41.734143 (+ 131us) client_negotiation.cc:240] Negotiation successful
0807 18:33:41.734308 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":100,"threads_started":1}
Aug 07 18:33:41 dist-test-slave-6k22 krb5kdc[19420](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127621, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 07 18:33:41 dist-test-slave-6k22 krb5kdc[19420](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.11.14.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:33:41 dist-test-slave-6k22 krb5kdc[19420](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.11.14.126@KRBTEST.COM, Server not found in Kerberos database
W20260807 18:33:41.758188 19517 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:33:41.752145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52786 (local address 127.11.14.126:45977)
0807 18:33:41.752290 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:41.752294 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:41.752307 (+ 13us) server_negotiation.cc:407] Connection header received
0807 18:33:41.752352 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:41.752354 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:41.752392 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:41.752468 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:33:41.753265 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:41.753774 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:41.754578 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:41.754779 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:41.754854 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:41.758058 (+ 3204us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52786: BlockingRecv error: recv got EOF from 127.0.0.1:52786 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 07 18:33:41 dist-test-slave-6k22 krb5kdc[19420](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127621, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.11.14.126@KRBTEST.COM
I20260807 18:33:41.767938 19517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:41.759712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52800 (local address 127.11.14.126:45977)
0807 18:33:41.759826 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:41.759829 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:41.759968 (+ 139us) server_negotiation.cc:407] Connection header received
0807 18:33:41.760058 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:41.760060 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:41.760096 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:41.760160 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:33:41.760704 (+ 544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:41.761176 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:41.761888 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:41.762083 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:41.762155 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:41.764536 (+ 2381us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:33:41.764539 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:33:41.764541 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:33:41.764584 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:33:41.766300 (+ 1716us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:41.766688 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:41.766690 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:41.766692 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:41.766765 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:41.767066 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:41.767068 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:41.767069 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:41.767230 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:33:41.767343 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:41.767644 (+ 301us) server_negotiation.cc:299] Negotiation successful
0807 18:33:41.767751 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260807 18:33:41.770710 19464 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52800:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 18:33:41.773958 19464 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 18:33:41.788985 19995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:41.782611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.67:35051 (local address 127.0.0.1:58600)
0807 18:33:41.783273 (+ 662us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:41.783322 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:41.783453 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:41.784445 (+ 992us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:41.784451 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:41.784473 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:33:41.784598 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:41.784604 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:41.786060 (+ 1456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:41.786063 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:41.787355 (+ 1292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:41.787365 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:41.788457 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:41.788460 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:41.788537 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:41.788568 (+ 31us) client_negotiation.cc:769] Sending connection context
0807 18:33:41.788709 (+ 141us) client_negotiation.cc:240] Negotiation successful
0807 18:33:41.788832 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":480,"mutex_wait_us":117,"thread_start_us":70,"threads_started":1}
I20260807 18:33:41.788996 19994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:41.782677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.66:43313 (local address 127.0.0.1:41810)
0807 18:33:41.783587 (+ 910us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:41.783601 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:41.784037 (+ 436us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:41.784737 (+ 700us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:41.784739 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:41.784757 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:33:41.784843 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:41.784849 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:41.786372 (+ 1523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:41.786376 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:41.787617 (+ 1241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:41.787623 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:41.788665 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:41.788671 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:41.788759 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:41.788789 (+ 30us) client_negotiation.cc:769] Sending connection context
0807 18:33:41.788850 (+ 61us) client_negotiation.cc:240] Negotiation successful
0807 18:33:41.788904 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":821,"mutex_wait_us":68,"thread_start_us":131,"threads_started":1}
I20260807 18:33:41.789455 19997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:41.783212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41810 (local address 127.11.14.66:43313)
0807 18:33:41.784160 (+ 948us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:41.784177 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:41.784202 (+ 25us) server_negotiation.cc:407] Connection header received
0807 18:33:41.784278 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:41.784282 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:41.784563 (+ 281us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:41.784992 (+ 429us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:33:41.785122 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:41.786232 (+ 1110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:41.787756 (+ 1524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:41.788538 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:41.788656 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:41.788811 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:41.788878 (+ 67us) server_negotiation.cc:299] Negotiation successful
0807 18:33:41.788971 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":842,"thread_start_us":101,"threads_started":1}
I20260807 18:33:41.789577 19996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:41.782760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58600 (local address 127.11.14.67:35051)
0807 18:33:41.783374 (+ 614us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:41.783386 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:41.783414 (+ 28us) server_negotiation.cc:407] Connection header received
0807 18:33:41.783872 (+ 458us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:41.783880 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:41.784105 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:41.784267 (+ 162us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:33:41.784737 (+ 470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:41.785930 (+ 1193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:41.787480 (+ 1550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:41.788324 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:41.788452 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:41.788632 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:41.789354 (+ 722us) server_negotiation.cc:299] Negotiation successful
0807 18:33:41.789434 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":478,"thread_start_us":126,"threads_started":1}
I20260807 18:33:41.790484 19998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:41.783293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.65:41543 (local address 127.0.0.1:45022)
0807 18:33:41.783819 (+ 526us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:41.783834 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:41.783917 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:41.784787 (+ 870us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:41.784791 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:41.784811 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:33:41.784931 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:41.784938 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:41.785992 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:41.785998 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:41.787609 (+ 1611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:41.787621 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:41.790025 (+ 2404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:41.790029 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:41.790125 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:41.790148 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 18:33:41.790199 (+ 51us) client_negotiation.cc:240] Negotiation successful
0807 18:33:41.790273 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":459,"mutex_wait_us":23,"thread_start_us":109,"threads_started":1}
I20260807 18:33:41.791150 19999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:41.783751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45022 (local address 127.11.14.65:41543)
0807 18:33:41.784311 (+ 560us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:41.784327 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:41.784356 (+ 29us) server_negotiation.cc:407] Connection header received
0807 18:33:41.784419 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:41.784422 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:41.784602 (+ 180us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:41.784808 (+ 206us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:33:41.785060 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:41.785880 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:41.788711 (+ 2831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:41.789542 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:41.790662 (+ 1120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:41.790799 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:41.790850 (+ 51us) server_negotiation.cc:299] Negotiation successful
0807 18:33:41.790937 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":442,"thread_start_us":99,"threads_started":1}
I20260807 18:33:41.792510 19754 tablet_service.cc:1467] Processing CreateTablet for tablet f7b9cbf89a144b90afaa0daaa2ec7143 (DEFAULT_TABLE table=test-table [id=606ec255ab9b48d8a91275757add6819]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:33:41.792520 19902 tablet_service.cc:1467] Processing CreateTablet for tablet f7b9cbf89a144b90afaa0daaa2ec7143 (DEFAULT_TABLE table=test-table [id=606ec255ab9b48d8a91275757add6819]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:33:41.793574 19902 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f7b9cbf89a144b90afaa0daaa2ec7143. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:33:41.793576 19754 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f7b9cbf89a144b90afaa0daaa2ec7143. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:33:41.794078 19610 tablet_service.cc:1467] Processing CreateTablet for tablet f7b9cbf89a144b90afaa0daaa2ec7143 (DEFAULT_TABLE table=test-table [id=606ec255ab9b48d8a91275757add6819]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:33:41.794994 19610 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f7b9cbf89a144b90afaa0daaa2ec7143. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:33:41.800427 20000 tablet_bootstrap.cc:492] T f7b9cbf89a144b90afaa0daaa2ec7143 P 841d14d044a3450dba7a52ace1d8f9f9: Bootstrap starting.
I20260807 18:33:41.801957 20001 tablet_bootstrap.cc:492] T f7b9cbf89a144b90afaa0daaa2ec7143 P 973fa32d73684f51949933c03a32ce8e: Bootstrap starting.
I20260807 18:33:41.802718 20002 tablet_bootstrap.cc:492] T f7b9cbf89a144b90afaa0daaa2ec7143 P b8b91f4589f343c88246ca60212c015a: Bootstrap starting.
I20260807 18:33:41.802774 20000 tablet_bootstrap.cc:654] T f7b9cbf89a144b90afaa0daaa2ec7143 P 841d14d044a3450dba7a52ace1d8f9f9: Neither blocks nor log segments found. Creating new log.
I20260807 18:33:41.803673 20000 log.cc:824] T f7b9cbf89a144b90afaa0daaa2ec7143 P 841d14d044a3450dba7a52ace1d8f9f9: Log is configured to *not* fsync() on all Append() calls
I20260807 18:33:41.804085 20001 tablet_bootstrap.cc:654] T f7b9cbf89a144b90afaa0daaa2ec7143 P 973fa32d73684f51949933c03a32ce8e: Neither blocks nor log segments found. Creating new log.
I20260807 18:33:41.804780 20002 tablet_bootstrap.cc:654] T f7b9cbf89a144b90afaa0daaa2ec7143 P b8b91f4589f343c88246ca60212c015a: Neither blocks nor log segments found. Creating new log.
I20260807 18:33:41.804939 20001 log.cc:824] T f7b9cbf89a144b90afaa0daaa2ec7143 P 973fa32d73684f51949933c03a32ce8e: Log is configured to *not* fsync() on all Append() calls
I20260807 18:33:41.805593 20002 log.cc:824] T f7b9cbf89a144b90afaa0daaa2ec7143 P b8b91f4589f343c88246ca60212c015a: Log is configured to *not* fsync() on all Append() calls
I20260807 18:33:41.806062 20000 tablet_bootstrap.cc:492] T f7b9cbf89a144b90afaa0daaa2ec7143 P 841d14d044a3450dba7a52ace1d8f9f9: No bootstrap required, opened a new log
I20260807 18:33:41.806300 20000 ts_tablet_manager.cc:1397] T f7b9cbf89a144b90afaa0daaa2ec7143 P 841d14d044a3450dba7a52ace1d8f9f9: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260807 18:33:41.807053 20001 tablet_bootstrap.cc:492] T f7b9cbf89a144b90afaa0daaa2ec7143 P 973fa32d73684f51949933c03a32ce8e: No bootstrap required, opened a new log
I20260807 18:33:41.807251 20001 ts_tablet_manager.cc:1397] T f7b9cbf89a144b90afaa0daaa2ec7143 P 973fa32d73684f51949933c03a32ce8e: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260807 18:33:41.807715 20002 tablet_bootstrap.cc:492] T f7b9cbf89a144b90afaa0daaa2ec7143 P b8b91f4589f343c88246ca60212c015a: No bootstrap required, opened a new log
I20260807 18:33:41.807895 20002 ts_tablet_manager.cc:1397] T f7b9cbf89a144b90afaa0daaa2ec7143 P b8b91f4589f343c88246ca60212c015a: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.003s
I20260807 18:33:41.809556 20000 raft_consensus.cc:348] T f7b9cbf89a144b90afaa0daaa2ec7143 P 841d14d044a3450dba7a52ace1d8f9f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "841d14d044a3450dba7a52ace1d8f9f9" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 35051 } } peers { permanent_uuid: "973fa32d73684f51949933c03a32ce8e" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 43313 } } peers { permanent_uuid: "b8b91f4589f343c88246ca60212c015a" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 41543 } }
I20260807 18:33:41.809801 20000 raft_consensus.cc:374] T f7b9cbf89a144b90afaa0daaa2ec7143 P 841d14d044a3450dba7a52ace1d8f9f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:33:41.809863 20000 raft_consensus.cc:729] T f7b9cbf89a144b90afaa0daaa2ec7143 P 841d14d044a3450dba7a52ace1d8f9f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 841d14d044a3450dba7a52ace1d8f9f9, State: Initialized, Role: FOLLOWER
I20260807 18:33:41.810309 20000 consensus_queue.cc:260] T f7b9cbf89a144b90afaa0daaa2ec7143 P 841d14d044a3450dba7a52ace1d8f9f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "841d14d044a3450dba7a52ace1d8f9f9" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 35051 } } peers { permanent_uuid: "973fa32d73684f51949933c03a32ce8e" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 43313 } } peers { permanent_uuid: "b8b91f4589f343c88246ca60212c015a" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 41543 } }
I20260807 18:33:41.811368 20000 ts_tablet_manager.cc:1428] T f7b9cbf89a144b90afaa0daaa2ec7143 P 841d14d044a3450dba7a52ace1d8f9f9: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260807 18:33:41.811218 20001 raft_consensus.cc:348] T f7b9cbf89a144b90afaa0daaa2ec7143 P 973fa32d73684f51949933c03a32ce8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "841d14d044a3450dba7a52ace1d8f9f9" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 35051 } } peers { permanent_uuid: "973fa32d73684f51949933c03a32ce8e" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 43313 } } peers { permanent_uuid: "b8b91f4589f343c88246ca60212c015a" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 41543 } }
I20260807 18:33:41.811455 20001 raft_consensus.cc:374] T f7b9cbf89a144b90afaa0daaa2ec7143 P 973fa32d73684f51949933c03a32ce8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:33:41.811513 20001 raft_consensus.cc:729] T f7b9cbf89a144b90afaa0daaa2ec7143 P 973fa32d73684f51949933c03a32ce8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 973fa32d73684f51949933c03a32ce8e, State: Initialized, Role: FOLLOWER
I20260807 18:33:41.811764 19971 heartbeater.cc:503] Master 127.11.14.126:45977 was elected leader, sending a full tablet report...
I20260807 18:33:41.811978 20001 consensus_queue.cc:260] T f7b9cbf89a144b90afaa0daaa2ec7143 P 973fa32d73684f51949933c03a32ce8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "841d14d044a3450dba7a52ace1d8f9f9" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 35051 } } peers { permanent_uuid: "973fa32d73684f51949933c03a32ce8e" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 43313 } } peers { permanent_uuid: "b8b91f4589f343c88246ca60212c015a" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 41543 } }
I20260807 18:33:41.812078 20002 raft_consensus.cc:348] T f7b9cbf89a144b90afaa0daaa2ec7143 P b8b91f4589f343c88246ca60212c015a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "841d14d044a3450dba7a52ace1d8f9f9" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 35051 } } peers { permanent_uuid: "973fa32d73684f51949933c03a32ce8e" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 43313 } } peers { permanent_uuid: "b8b91f4589f343c88246ca60212c015a" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 41543 } }
I20260807 18:33:41.812327 20002 raft_consensus.cc:374] T f7b9cbf89a144b90afaa0daaa2ec7143 P b8b91f4589f343c88246ca60212c015a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:33:41.812409 20002 raft_consensus.cc:729] T f7b9cbf89a144b90afaa0daaa2ec7143 P b8b91f4589f343c88246ca60212c015a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8b91f4589f343c88246ca60212c015a, State: Initialized, Role: FOLLOWER
I20260807 18:33:41.812666 19823 heartbeater.cc:503] Master 127.11.14.126:45977 was elected leader, sending a full tablet report...
I20260807 18:33:41.812783 20002 consensus_queue.cc:260] T f7b9cbf89a144b90afaa0daaa2ec7143 P b8b91f4589f343c88246ca60212c015a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "841d14d044a3450dba7a52ace1d8f9f9" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 35051 } } peers { permanent_uuid: "973fa32d73684f51949933c03a32ce8e" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 43313 } } peers { permanent_uuid: "b8b91f4589f343c88246ca60212c015a" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 41543 } }
I20260807 18:33:41.812942 20001 ts_tablet_manager.cc:1428] T f7b9cbf89a144b90afaa0daaa2ec7143 P 973fa32d73684f51949933c03a32ce8e: Time spent starting tablet: real 0.006s user 0.003s sys 0.003s
I20260807 18:33:41.813639 20002 ts_tablet_manager.cc:1428] T f7b9cbf89a144b90afaa0daaa2ec7143 P b8b91f4589f343c88246ca60212c015a: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260807 18:33:41.813923 19678 heartbeater.cc:503] Master 127.11.14.126:45977 was elected leader, sending a full tablet report...
I20260807 18:33:41.889744 20006 raft_consensus.cc:482] T f7b9cbf89a144b90afaa0daaa2ec7143 P 841d14d044a3450dba7a52ace1d8f9f9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:33:41.890030 20006 raft_consensus.cc:504] T f7b9cbf89a144b90afaa0daaa2ec7143 P 841d14d044a3450dba7a52ace1d8f9f9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "841d14d044a3450dba7a52ace1d8f9f9" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 35051 } } peers { permanent_uuid: "973fa32d73684f51949933c03a32ce8e" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 43313 } } peers { permanent_uuid: "b8b91f4589f343c88246ca60212c015a" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 41543 } }
I20260807 18:33:41.891297 20006 leader_election.cc:288] T f7b9cbf89a144b90afaa0daaa2ec7143 P 841d14d044a3450dba7a52ace1d8f9f9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 973fa32d73684f51949933c03a32ce8e (127.11.14.66:43313), b8b91f4589f343c88246ca60212c015a (127.11.14.65:41543)
I20260807 18:33:41.895855 19997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:41.891659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.67:38069 (local address 127.11.14.66:43313)
0807 18:33:41.891803 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:41.891808 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:41.891827 (+ 19us) server_negotiation.cc:407] Connection header received
0807 18:33:41.891937 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:41.891943 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:41.891991 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:41.892103 (+ 112us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:33:41.892423 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:41.893253 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:41.894681 (+ 1428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:41.895285 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:41.895417 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:41.895487 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:41.895689 (+ 202us) server_negotiation.cc:299] Negotiation successful
0807 18:33:41.895752 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260807 18:33:41.895864 19859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:41.891529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.66:43313 (local address 127.11.14.67:38069)
0807 18:33:41.891686 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:41.891703 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:41.891824 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:41.892135 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:41.892139 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:41.892160 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:33:41.892296 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:41.892302 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:41.893395 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:41.893398 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:41.894550 (+ 1152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:41.894560 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:41.895412 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:41.895415 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:41.895527 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:41.895553 (+ 26us) client_negotiation.cc:769] Sending connection context
0807 18:33:41.895654 (+ 101us) client_negotiation.cc:240] Negotiation successful
0807 18:33:41.895732 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":68}
I20260807 18:33:41.896634 20009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:41.891920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.65:41543 (local address 127.11.14.67:46775)
0807 18:33:41.892325 (+ 405us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:41.892339 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:41.892466 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:41.892761 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:41.892765 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:41.892784 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:33:41.892918 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:41.892924 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:41.894310 (+ 1386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:41.894313 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:41.895412 (+ 1099us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:41.895423 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:41.896336 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:41.896340 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:41.896412 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:41.896431 (+ 19us) client_negotiation.cc:769] Sending connection context
0807 18:33:41.896494 (+ 63us) client_negotiation.cc:240] Negotiation successful
0807 18:33:41.896542 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":343,"thread_start_us":170,"threads_started":1}
I20260807 18:33:41.896579 19773 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f7b9cbf89a144b90afaa0daaa2ec7143" candidate_uuid: "841d14d044a3450dba7a52ace1d8f9f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "973fa32d73684f51949933c03a32ce8e" is_pre_election: true
I20260807 18:33:41.896713 19999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:41.891719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.67:46775 (local address 127.11.14.65:41543)
0807 18:33:41.891893 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:41.891898 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:41.892448 (+ 550us) server_negotiation.cc:407] Connection header received
0807 18:33:41.892566 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:41.892568 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:41.892637 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:41.892712 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:33:41.893426 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:41.894185 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:41.895532 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:41.896183 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:41.896326 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:41.896372 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:41.896529 (+ 157us) server_negotiation.cc:299] Negotiation successful
0807 18:33:41.896599 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260807 18:33:41.896912 19773 raft_consensus.cc:2454] T f7b9cbf89a144b90afaa0daaa2ec7143 P 973fa32d73684f51949933c03a32ce8e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 841d14d044a3450dba7a52ace1d8f9f9 in term 0.
I20260807 18:33:41.897436 19851 leader_election.cc:302] T f7b9cbf89a144b90afaa0daaa2ec7143 P 841d14d044a3450dba7a52ace1d8f9f9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 841d14d044a3450dba7a52ace1d8f9f9, 973fa32d73684f51949933c03a32ce8e; no voters:
I20260807 18:33:41.897437 19630 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f7b9cbf89a144b90afaa0daaa2ec7143" candidate_uuid: "841d14d044a3450dba7a52ace1d8f9f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8b91f4589f343c88246ca60212c015a" is_pre_election: true
I20260807 18:33:41.897717 20006 raft_consensus.cc:2781] T f7b9cbf89a144b90afaa0daaa2ec7143 P 841d14d044a3450dba7a52ace1d8f9f9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:33:41.897758 19630 raft_consensus.cc:2454] T f7b9cbf89a144b90afaa0daaa2ec7143 P b8b91f4589f343c88246ca60212c015a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 841d14d044a3450dba7a52ace1d8f9f9 in term 0.
I20260807 18:33:41.897825 20006 raft_consensus.cc:482] T f7b9cbf89a144b90afaa0daaa2ec7143 P 841d14d044a3450dba7a52ace1d8f9f9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:33:41.897881 20006 raft_consensus.cc:3037] T f7b9cbf89a144b90afaa0daaa2ec7143 P 841d14d044a3450dba7a52ace1d8f9f9 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:33:41.898975 20006 raft_consensus.cc:504] T f7b9cbf89a144b90afaa0daaa2ec7143 P 841d14d044a3450dba7a52ace1d8f9f9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "841d14d044a3450dba7a52ace1d8f9f9" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 35051 } } peers { permanent_uuid: "973fa32d73684f51949933c03a32ce8e" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 43313 } } peers { permanent_uuid: "b8b91f4589f343c88246ca60212c015a" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 41543 } }
I20260807 18:33:41.899269 20006 leader_election.cc:288] T f7b9cbf89a144b90afaa0daaa2ec7143 P 841d14d044a3450dba7a52ace1d8f9f9 [CANDIDATE]: Term 1 election: Requested vote from peers 973fa32d73684f51949933c03a32ce8e (127.11.14.66:43313), b8b91f4589f343c88246ca60212c015a (127.11.14.65:41543)
I20260807 18:33:41.899652 19773 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f7b9cbf89a144b90afaa0daaa2ec7143" candidate_uuid: "841d14d044a3450dba7a52ace1d8f9f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "973fa32d73684f51949933c03a32ce8e"
I20260807 18:33:41.899652 19630 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f7b9cbf89a144b90afaa0daaa2ec7143" candidate_uuid: "841d14d044a3450dba7a52ace1d8f9f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8b91f4589f343c88246ca60212c015a"
I20260807 18:33:41.899775 19630 raft_consensus.cc:3037] T f7b9cbf89a144b90afaa0daaa2ec7143 P b8b91f4589f343c88246ca60212c015a [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:33:41.899775 19773 raft_consensus.cc:3037] T f7b9cbf89a144b90afaa0daaa2ec7143 P 973fa32d73684f51949933c03a32ce8e [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:33:41.901141 19630 raft_consensus.cc:2454] T f7b9cbf89a144b90afaa0daaa2ec7143 P b8b91f4589f343c88246ca60212c015a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 841d14d044a3450dba7a52ace1d8f9f9 in term 1.
I20260807 18:33:41.901525 19852 leader_election.cc:302] T f7b9cbf89a144b90afaa0daaa2ec7143 P 841d14d044a3450dba7a52ace1d8f9f9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 841d14d044a3450dba7a52ace1d8f9f9, b8b91f4589f343c88246ca60212c015a; no voters:
I20260807 18:33:41.901589 19773 raft_consensus.cc:2454] T f7b9cbf89a144b90afaa0daaa2ec7143 P 973fa32d73684f51949933c03a32ce8e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 841d14d044a3450dba7a52ace1d8f9f9 in term 1.
I20260807 18:33:41.901721 20006 raft_consensus.cc:2781] T f7b9cbf89a144b90afaa0daaa2ec7143 P 841d14d044a3450dba7a52ace1d8f9f9 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:33:41.901942 20006 raft_consensus.cc:686] T f7b9cbf89a144b90afaa0daaa2ec7143 P 841d14d044a3450dba7a52ace1d8f9f9 [term 1 LEADER]: Becoming Leader. State: Replica: 841d14d044a3450dba7a52ace1d8f9f9, State: Running, Role: LEADER
I20260807 18:33:41.902352 20006 consensus_queue.cc:237] T f7b9cbf89a144b90afaa0daaa2ec7143 P 841d14d044a3450dba7a52ace1d8f9f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "841d14d044a3450dba7a52ace1d8f9f9" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 35051 } } peers { permanent_uuid: "973fa32d73684f51949933c03a32ce8e" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 43313 } } peers { permanent_uuid: "b8b91f4589f343c88246ca60212c015a" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 41543 } }
I20260807 18:33:41.905689 19465 catalog_manager.cc:5495] T f7b9cbf89a144b90afaa0daaa2ec7143 P 841d14d044a3450dba7a52ace1d8f9f9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 841d14d044a3450dba7a52ace1d8f9f9 (127.11.14.67). New cstate: current_term: 1 leader_uuid: "841d14d044a3450dba7a52ace1d8f9f9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "841d14d044a3450dba7a52ace1d8f9f9" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 35051 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "973fa32d73684f51949933c03a32ce8e" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 43313 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b8b91f4589f343c88246ca60212c015a" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 41543 } health_report { overall_health: UNKNOWN } } }
I20260807 18:33:41.940809 19996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:41.937093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58614 (local address 127.11.14.67:35051)
0807 18:33:41.937224 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:41.937228 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:41.937245 (+ 17us) server_negotiation.cc:407] Connection header received
0807 18:33:41.937379 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:41.937381 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:41.937437 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:41.937555 (+ 118us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:33:41.937821 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:41.938338 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:41.939297 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:41.939529 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:41.939640 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:41.939950 (+ 310us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:33:41.940370 (+ 420us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:33:41.940457 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:41.940571 (+ 114us) server_negotiation.cc:299] Negotiation successful
0807 18:33:41.940629 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260807 18:33:41.947254 19630 raft_consensus.cc:1261] T f7b9cbf89a144b90afaa0daaa2ec7143 P b8b91f4589f343c88246ca60212c015a [term 1 FOLLOWER]: Refusing update from remote peer 841d14d044a3450dba7a52ace1d8f9f9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:33:41.947273 19773 raft_consensus.cc:1261] T f7b9cbf89a144b90afaa0daaa2ec7143 P 973fa32d73684f51949933c03a32ce8e [term 1 FOLLOWER]: Refusing update from remote peer 841d14d044a3450dba7a52ace1d8f9f9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:33:41.948033 20010 consensus_queue.cc:1035] T f7b9cbf89a144b90afaa0daaa2ec7143 P 841d14d044a3450dba7a52ace1d8f9f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "973fa32d73684f51949933c03a32ce8e" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 43313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:33:41.948225 20006 consensus_queue.cc:1035] T f7b9cbf89a144b90afaa0daaa2ec7143 P 841d14d044a3450dba7a52ace1d8f9f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8b91f4589f343c88246ca60212c015a" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 41543 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:33:41.955510 20013 mvcc.cc:205] Tried to move back new op lower bound from 7315978739488227328 to 7315978739314393088. Current Snapshot: MvccSnapshot[applied={T|T < 7315978739488227328}]
I20260807 18:33:41.957720 20014 mvcc.cc:205] Tried to move back new op lower bound from 7315978739488227328 to 7315978739314393088. Current Snapshot: MvccSnapshot[applied={T|T < 7315978739488227328}]
I20260807 18:33:41.958089 20015 mvcc.cc:205] Tried to move back new op lower bound from 7315978739488227328 to 7315978739314393088. Current Snapshot: MvccSnapshot[applied={T|T < 7315978739488227328}]
I20260807 18:33:41.968317 19465 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52800:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:33:41.968541 19465 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52800:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:33:41.970983 19465 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8482b9c6ca944b908d05b969396896b8: Sending DeleteTablet for 3 replicas of tablet f7b9cbf89a144b90afaa0daaa2ec7143
I20260807 18:33:41.971822 19754 tablet_service.cc:1514] Processing DeleteTablet for tablet f7b9cbf89a144b90afaa0daaa2ec7143 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:33:41 UTC) from {username='oryx', principal='oryx/127.11.14.126@KRBTEST.COM'} at 127.0.0.1:41810
I20260807 18:33:41.971897 19902 tablet_service.cc:1514] Processing DeleteTablet for tablet f7b9cbf89a144b90afaa0daaa2ec7143 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:33:41 UTC) from {username='oryx', principal='oryx/127.11.14.126@KRBTEST.COM'} at 127.0.0.1:58600
I20260807 18:33:41.972772 20024 tablet_replica.cc:330] T f7b9cbf89a144b90afaa0daaa2ec7143 P 841d14d044a3450dba7a52ace1d8f9f9: stopping tablet replica
I20260807 18:33:41.972766 19610 tablet_service.cc:1514] Processing DeleteTablet for tablet f7b9cbf89a144b90afaa0daaa2ec7143 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:33:41 UTC) from {username='oryx', principal='oryx/127.11.14.126@KRBTEST.COM'} at 127.0.0.1:45022
I20260807 18:33:41.973043 19465 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52800:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 18:33:41.973244 20025 tablet_replica.cc:330] T f7b9cbf89a144b90afaa0daaa2ec7143 P b8b91f4589f343c88246ca60212c015a: stopping tablet replica
W20260807 18:33:41.973470 19465 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 18:33:41.973562 20025 raft_consensus.cc:2229] T f7b9cbf89a144b90afaa0daaa2ec7143 P b8b91f4589f343c88246ca60212c015a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:33:41.973605 20024 raft_consensus.cc:2229] T f7b9cbf89a144b90afaa0daaa2ec7143 P 841d14d044a3450dba7a52ace1d8f9f9 [term 1 LEADER]: Raft consensus shutting down.
I20260807 18:33:41.973987 20025 raft_consensus.cc:2258] T f7b9cbf89a144b90afaa0daaa2ec7143 P b8b91f4589f343c88246ca60212c015a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:33:41.974296 20024 raft_consensus.cc:2258] T f7b9cbf89a144b90afaa0daaa2ec7143 P 841d14d044a3450dba7a52ace1d8f9f9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:33:41.974354 20022 tablet_replica.cc:330] T f7b9cbf89a144b90afaa0daaa2ec7143 P 973fa32d73684f51949933c03a32ce8e: stopping tablet replica
I20260807 18:33:41.975103 20022 raft_consensus.cc:2229] T f7b9cbf89a144b90afaa0daaa2ec7143 P 973fa32d73684f51949933c03a32ce8e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:33:41.975399 20022 raft_consensus.cc:2258] T f7b9cbf89a144b90afaa0daaa2ec7143 P 973fa32d73684f51949933c03a32ce8e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:33:41.977074 20022 ts_tablet_manager.cc:1905] T f7b9cbf89a144b90afaa0daaa2ec7143 P 973fa32d73684f51949933c03a32ce8e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:33:41.977342 20024 ts_tablet_manager.cc:1905] T f7b9cbf89a144b90afaa0daaa2ec7143 P 841d14d044a3450dba7a52ace1d8f9f9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:33:41.979094 19610 tablet_service.cc:1467] Processing CreateTablet for tablet 49a0eda694ac4e358e65e146714b9d82 (DEFAULT_TABLE table=test-table [id=d4132ac9baa847dbb5cb305851a9fd70]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:33:41.979373 19610 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 49a0eda694ac4e358e65e146714b9d82. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:33:41.979835 20025 ts_tablet_manager.cc:1905] T f7b9cbf89a144b90afaa0daaa2ec7143 P b8b91f4589f343c88246ca60212c015a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:33:41.980356 20022 ts_tablet_manager.cc:1918] T f7b9cbf89a144b90afaa0daaa2ec7143 P 973fa32d73684f51949933c03a32ce8e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:33:41.980449 20022 log.cc:1196] T f7b9cbf89a144b90afaa0daaa2ec7143 P 973fa32d73684f51949933c03a32ce8e: Deleting WAL directory at /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127579900523-11321-0/minicluster-data/ts-1/wal/wals/f7b9cbf89a144b90afaa0daaa2ec7143
I20260807 18:33:41.980777 20022 ts_tablet_manager.cc:1939] T f7b9cbf89a144b90afaa0daaa2ec7143 P 973fa32d73684f51949933c03a32ce8e: Deleting consensus metadata
I20260807 18:33:41.981859 20002 tablet_bootstrap.cc:492] T 49a0eda694ac4e358e65e146714b9d82 P b8b91f4589f343c88246ca60212c015a: Bootstrap starting.
I20260807 18:33:41.982054 19902 tablet_service.cc:1467] Processing CreateTablet for tablet 49a0eda694ac4e358e65e146714b9d82 (DEFAULT_TABLE table=test-table [id=d4132ac9baa847dbb5cb305851a9fd70]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:33:41.982295 19902 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 49a0eda694ac4e358e65e146714b9d82. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:33:41.982867 20002 tablet_bootstrap.cc:654] T 49a0eda694ac4e358e65e146714b9d82 P b8b91f4589f343c88246ca60212c015a: Neither blocks nor log segments found. Creating new log.
I20260807 18:33:41.983300 20025 ts_tablet_manager.cc:1918] T f7b9cbf89a144b90afaa0daaa2ec7143 P b8b91f4589f343c88246ca60212c015a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:33:41.983845 20025 log.cc:1196] T f7b9cbf89a144b90afaa0daaa2ec7143 P b8b91f4589f343c88246ca60212c015a: Deleting WAL directory at /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127579900523-11321-0/minicluster-data/ts-0/wal/wals/f7b9cbf89a144b90afaa0daaa2ec7143
I20260807 18:33:41.984241 20025 ts_tablet_manager.cc:1939] T f7b9cbf89a144b90afaa0daaa2ec7143 P b8b91f4589f343c88246ca60212c015a: Deleting consensus metadata
I20260807 18:33:41.984735 20000 tablet_bootstrap.cc:492] T 49a0eda694ac4e358e65e146714b9d82 P 841d14d044a3450dba7a52ace1d8f9f9: Bootstrap starting.
I20260807 18:33:41.984915 19754 tablet_service.cc:1467] Processing CreateTablet for tablet 49a0eda694ac4e358e65e146714b9d82 (DEFAULT_TABLE table=test-table [id=d4132ac9baa847dbb5cb305851a9fd70]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:33:41.985186 19754 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 49a0eda694ac4e358e65e146714b9d82. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:33:41.985700 20000 tablet_bootstrap.cc:654] T 49a0eda694ac4e358e65e146714b9d82 P 841d14d044a3450dba7a52ace1d8f9f9: Neither blocks nor log segments found. Creating new log.
I20260807 18:33:41.986886 20000 tablet_bootstrap.cc:492] T 49a0eda694ac4e358e65e146714b9d82 P 841d14d044a3450dba7a52ace1d8f9f9: No bootstrap required, opened a new log
I20260807 18:33:41.986974 20000 ts_tablet_manager.cc:1397] T 49a0eda694ac4e358e65e146714b9d82 P 841d14d044a3450dba7a52ace1d8f9f9: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260807 18:33:41.987437 20001 tablet_bootstrap.cc:492] T 49a0eda694ac4e358e65e146714b9d82 P 973fa32d73684f51949933c03a32ce8e: Bootstrap starting.
I20260807 18:33:41.987442 20000 raft_consensus.cc:348] T 49a0eda694ac4e358e65e146714b9d82 P 841d14d044a3450dba7a52ace1d8f9f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8b91f4589f343c88246ca60212c015a" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 41543 } } peers { permanent_uuid: "973fa32d73684f51949933c03a32ce8e" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 43313 } } peers { permanent_uuid: "841d14d044a3450dba7a52ace1d8f9f9" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 35051 } }
I20260807 18:33:41.987550 20000 raft_consensus.cc:374] T 49a0eda694ac4e358e65e146714b9d82 P 841d14d044a3450dba7a52ace1d8f9f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:33:41.988090 20000 raft_consensus.cc:729] T 49a0eda694ac4e358e65e146714b9d82 P 841d14d044a3450dba7a52ace1d8f9f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 841d14d044a3450dba7a52ace1d8f9f9, State: Initialized, Role: FOLLOWER
I20260807 18:33:41.988113 19450 catalog_manager.cc:4841] TS 973fa32d73684f51949933c03a32ce8e (127.11.14.66:43313): tablet f7b9cbf89a144b90afaa0daaa2ec7143 (table test-table [id=606ec255ab9b48d8a91275757add6819]) successfully deleted
I20260807 18:33:41.988225 20000 consensus_queue.cc:260] T 49a0eda694ac4e358e65e146714b9d82 P 841d14d044a3450dba7a52ace1d8f9f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8b91f4589f343c88246ca60212c015a" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 41543 } } peers { permanent_uuid: "973fa32d73684f51949933c03a32ce8e" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 43313 } } peers { permanent_uuid: "841d14d044a3450dba7a52ace1d8f9f9" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 35051 } }
I20260807 18:33:41.988479 20001 tablet_bootstrap.cc:654] T 49a0eda694ac4e358e65e146714b9d82 P 973fa32d73684f51949933c03a32ce8e: Neither blocks nor log segments found. Creating new log.
I20260807 18:33:41.988524 20000 ts_tablet_manager.cc:1428] T 49a0eda694ac4e358e65e146714b9d82 P 841d14d044a3450dba7a52ace1d8f9f9: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260807 18:33:41.989171 19451 catalog_manager.cc:4841] TS b8b91f4589f343c88246ca60212c015a (127.11.14.65:41543): tablet f7b9cbf89a144b90afaa0daaa2ec7143 (table test-table [id=606ec255ab9b48d8a91275757add6819]) successfully deleted
I20260807 18:33:41.989738 20001 tablet_bootstrap.cc:492] T 49a0eda694ac4e358e65e146714b9d82 P 973fa32d73684f51949933c03a32ce8e: No bootstrap required, opened a new log
I20260807 18:33:41.989846 20001 ts_tablet_manager.cc:1397] T 49a0eda694ac4e358e65e146714b9d82 P 973fa32d73684f51949933c03a32ce8e: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260807 18:33:41.990278 20001 raft_consensus.cc:348] T 49a0eda694ac4e358e65e146714b9d82 P 973fa32d73684f51949933c03a32ce8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8b91f4589f343c88246ca60212c015a" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 41543 } } peers { permanent_uuid: "973fa32d73684f51949933c03a32ce8e" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 43313 } } peers { permanent_uuid: "841d14d044a3450dba7a52ace1d8f9f9" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 35051 } }
I20260807 18:33:41.990406 20001 raft_consensus.cc:374] T 49a0eda694ac4e358e65e146714b9d82 P 973fa32d73684f51949933c03a32ce8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:33:41.990448 20001 raft_consensus.cc:729] T 49a0eda694ac4e358e65e146714b9d82 P 973fa32d73684f51949933c03a32ce8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 973fa32d73684f51949933c03a32ce8e, State: Initialized, Role: FOLLOWER
I20260807 18:33:41.990573 20001 consensus_queue.cc:260] T 49a0eda694ac4e358e65e146714b9d82 P 973fa32d73684f51949933c03a32ce8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8b91f4589f343c88246ca60212c015a" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 41543 } } peers { permanent_uuid: "973fa32d73684f51949933c03a32ce8e" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 43313 } } peers { permanent_uuid: "841d14d044a3450dba7a52ace1d8f9f9" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 35051 } }
I20260807 18:33:41.990854 20001 ts_tablet_manager.cc:1428] T 49a0eda694ac4e358e65e146714b9d82 P 973fa32d73684f51949933c03a32ce8e: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260807 18:33:41.993263 20024 ts_tablet_manager.cc:1918] T f7b9cbf89a144b90afaa0daaa2ec7143 P 841d14d044a3450dba7a52ace1d8f9f9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:33:41.993410 20024 log.cc:1196] T f7b9cbf89a144b90afaa0daaa2ec7143 P 841d14d044a3450dba7a52ace1d8f9f9: Deleting WAL directory at /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127579900523-11321-0/minicluster-data/ts-2/wal/wals/f7b9cbf89a144b90afaa0daaa2ec7143
I20260807 18:33:41.993885 20024 ts_tablet_manager.cc:1939] T f7b9cbf89a144b90afaa0daaa2ec7143 P 841d14d044a3450dba7a52ace1d8f9f9: Deleting consensus metadata
I20260807 18:33:41.995048 19451 catalog_manager.cc:4841] TS 841d14d044a3450dba7a52ace1d8f9f9 (127.11.14.67:35051): tablet f7b9cbf89a144b90afaa0daaa2ec7143 (table test-table [id=606ec255ab9b48d8a91275757add6819]) successfully deleted
I20260807 18:33:41.995772 20002 tablet_bootstrap.cc:492] T 49a0eda694ac4e358e65e146714b9d82 P b8b91f4589f343c88246ca60212c015a: No bootstrap required, opened a new log
I20260807 18:33:41.995899 20002 ts_tablet_manager.cc:1397] T 49a0eda694ac4e358e65e146714b9d82 P b8b91f4589f343c88246ca60212c015a: Time spent bootstrapping tablet: real 0.014s user 0.001s sys 0.001s
I20260807 18:33:41.996587 20002 raft_consensus.cc:348] T 49a0eda694ac4e358e65e146714b9d82 P b8b91f4589f343c88246ca60212c015a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8b91f4589f343c88246ca60212c015a" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 41543 } } peers { permanent_uuid: "973fa32d73684f51949933c03a32ce8e" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 43313 } } peers { permanent_uuid: "841d14d044a3450dba7a52ace1d8f9f9" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 35051 } }
I20260807 18:33:41.996735 20002 raft_consensus.cc:374] T 49a0eda694ac4e358e65e146714b9d82 P b8b91f4589f343c88246ca60212c015a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:33:41.996779 20002 raft_consensus.cc:729] T 49a0eda694ac4e358e65e146714b9d82 P b8b91f4589f343c88246ca60212c015a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8b91f4589f343c88246ca60212c015a, State: Initialized, Role: FOLLOWER
I20260807 18:33:41.996942 20002 consensus_queue.cc:260] T 49a0eda694ac4e358e65e146714b9d82 P b8b91f4589f343c88246ca60212c015a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8b91f4589f343c88246ca60212c015a" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 41543 } } peers { permanent_uuid: "973fa32d73684f51949933c03a32ce8e" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 43313 } } peers { permanent_uuid: "841d14d044a3450dba7a52ace1d8f9f9" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 35051 } }
I20260807 18:33:41.997231 20002 ts_tablet_manager.cc:1428] T 49a0eda694ac4e358e65e146714b9d82 P b8b91f4589f343c88246ca60212c015a: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
W20260807 18:33:42.003172 19824 tablet.cc:2369] T 49a0eda694ac4e358e65e146714b9d82 P 973fa32d73684f51949933c03a32ce8e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 18:33:42.033289 19679 tablet.cc:2369] T 49a0eda694ac4e358e65e146714b9d82 P b8b91f4589f343c88246ca60212c015a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:33:42.129565 20010 raft_consensus.cc:482] T 49a0eda694ac4e358e65e146714b9d82 P 841d14d044a3450dba7a52ace1d8f9f9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:33:42.129738 20010 raft_consensus.cc:504] T 49a0eda694ac4e358e65e146714b9d82 P 841d14d044a3450dba7a52ace1d8f9f9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8b91f4589f343c88246ca60212c015a" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 41543 } } peers { permanent_uuid: "973fa32d73684f51949933c03a32ce8e" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 43313 } } peers { permanent_uuid: "841d14d044a3450dba7a52ace1d8f9f9" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 35051 } }
I20260807 18:33:42.130240 20010 leader_election.cc:288] T 49a0eda694ac4e358e65e146714b9d82 P 841d14d044a3450dba7a52ace1d8f9f9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8b91f4589f343c88246ca60212c015a (127.11.14.65:41543), 973fa32d73684f51949933c03a32ce8e (127.11.14.66:43313)
I20260807 18:33:42.130686 19630 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "49a0eda694ac4e358e65e146714b9d82" candidate_uuid: "841d14d044a3450dba7a52ace1d8f9f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8b91f4589f343c88246ca60212c015a" is_pre_election: true
I20260807 18:33:42.130684 19773 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "49a0eda694ac4e358e65e146714b9d82" candidate_uuid: "841d14d044a3450dba7a52ace1d8f9f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "973fa32d73684f51949933c03a32ce8e" is_pre_election: true
I20260807 18:33:42.130870 19630 raft_consensus.cc:2454] T 49a0eda694ac4e358e65e146714b9d82 P b8b91f4589f343c88246ca60212c015a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 841d14d044a3450dba7a52ace1d8f9f9 in term 0.
I20260807 18:33:42.130877 19773 raft_consensus.cc:2454] T 49a0eda694ac4e358e65e146714b9d82 P 973fa32d73684f51949933c03a32ce8e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 841d14d044a3450dba7a52ace1d8f9f9 in term 0.
I20260807 18:33:42.131181 19852 leader_election.cc:302] T 49a0eda694ac4e358e65e146714b9d82 P 841d14d044a3450dba7a52ace1d8f9f9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 841d14d044a3450dba7a52ace1d8f9f9, b8b91f4589f343c88246ca60212c015a; no voters:
I20260807 18:33:42.131381 20010 raft_consensus.cc:2781] T 49a0eda694ac4e358e65e146714b9d82 P 841d14d044a3450dba7a52ace1d8f9f9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:33:42.131438 20010 raft_consensus.cc:482] T 49a0eda694ac4e358e65e146714b9d82 P 841d14d044a3450dba7a52ace1d8f9f9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:33:42.131465 20010 raft_consensus.cc:3037] T 49a0eda694ac4e358e65e146714b9d82 P 841d14d044a3450dba7a52ace1d8f9f9 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:33:42.132494 20010 raft_consensus.cc:504] T 49a0eda694ac4e358e65e146714b9d82 P 841d14d044a3450dba7a52ace1d8f9f9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8b91f4589f343c88246ca60212c015a" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 41543 } } peers { permanent_uuid: "973fa32d73684f51949933c03a32ce8e" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 43313 } } peers { permanent_uuid: "841d14d044a3450dba7a52ace1d8f9f9" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 35051 } }
I20260807 18:33:42.132866 20010 leader_election.cc:288] T 49a0eda694ac4e358e65e146714b9d82 P 841d14d044a3450dba7a52ace1d8f9f9 [CANDIDATE]: Term 1 election: Requested vote from peers b8b91f4589f343c88246ca60212c015a (127.11.14.65:41543), 973fa32d73684f51949933c03a32ce8e (127.11.14.66:43313)
I20260807 18:33:42.133229 19630 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "49a0eda694ac4e358e65e146714b9d82" candidate_uuid: "841d14d044a3450dba7a52ace1d8f9f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8b91f4589f343c88246ca60212c015a"
I20260807 18:33:42.133296 19773 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "49a0eda694ac4e358e65e146714b9d82" candidate_uuid: "841d14d044a3450dba7a52ace1d8f9f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "973fa32d73684f51949933c03a32ce8e"
I20260807 18:33:42.133379 19630 raft_consensus.cc:3037] T 49a0eda694ac4e358e65e146714b9d82 P b8b91f4589f343c88246ca60212c015a [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:33:42.133443 19773 raft_consensus.cc:3037] T 49a0eda694ac4e358e65e146714b9d82 P 973fa32d73684f51949933c03a32ce8e [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:33:42.134383 19630 raft_consensus.cc:2454] T 49a0eda694ac4e358e65e146714b9d82 P b8b91f4589f343c88246ca60212c015a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 841d14d044a3450dba7a52ace1d8f9f9 in term 1.
I20260807 18:33:42.134689 19852 leader_election.cc:302] T 49a0eda694ac4e358e65e146714b9d82 P 841d14d044a3450dba7a52ace1d8f9f9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 841d14d044a3450dba7a52ace1d8f9f9, b8b91f4589f343c88246ca60212c015a; no voters:
I20260807 18:33:42.134761 19773 raft_consensus.cc:2454] T 49a0eda694ac4e358e65e146714b9d82 P 973fa32d73684f51949933c03a32ce8e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 841d14d044a3450dba7a52ace1d8f9f9 in term 1.
I20260807 18:33:42.134891 20010 raft_consensus.cc:2781] T 49a0eda694ac4e358e65e146714b9d82 P 841d14d044a3450dba7a52ace1d8f9f9 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:33:42.135006 20010 raft_consensus.cc:686] T 49a0eda694ac4e358e65e146714b9d82 P 841d14d044a3450dba7a52ace1d8f9f9 [term 1 LEADER]: Becoming Leader. State: Replica: 841d14d044a3450dba7a52ace1d8f9f9, State: Running, Role: LEADER
I20260807 18:33:42.135154 20010 consensus_queue.cc:237] T 49a0eda694ac4e358e65e146714b9d82 P 841d14d044a3450dba7a52ace1d8f9f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8b91f4589f343c88246ca60212c015a" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 41543 } } peers { permanent_uuid: "973fa32d73684f51949933c03a32ce8e" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 43313 } } peers { permanent_uuid: "841d14d044a3450dba7a52ace1d8f9f9" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 35051 } }
I20260807 18:33:42.136906 19464 catalog_manager.cc:5495] T 49a0eda694ac4e358e65e146714b9d82 P 841d14d044a3450dba7a52ace1d8f9f9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 841d14d044a3450dba7a52ace1d8f9f9 (127.11.14.67). New cstate: current_term: 1 leader_uuid: "841d14d044a3450dba7a52ace1d8f9f9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8b91f4589f343c88246ca60212c015a" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 41543 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "973fa32d73684f51949933c03a32ce8e" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 43313 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "841d14d044a3450dba7a52ace1d8f9f9" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 35051 } health_report { overall_health: HEALTHY } } }
Aug 07 18:33:42 dist-test-slave-6k22 krb5kdc[19420](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127620, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.14.126@KRBTEST.COM for oryx/127.11.14.126@KRBTEST.COM
I20260807 18:33:42.197805 19517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:42.189319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52808 (local address 127.11.14.126:45977)
0807 18:33:42.189470 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:42.189475 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:42.189658 (+ 183us) server_negotiation.cc:407] Connection header received
0807 18:33:42.189755 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:42.189758 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:42.189799 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:42.189868 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:33:42.190521 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:42.191025 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:42.191845 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:42.192076 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:42.192205 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:42.194575 (+ 2370us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:33:42.194579 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:33:42.194588 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:33:42.194608 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:33:42.196033 (+ 1425us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:42.196504 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:42.196509 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:42.196511 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:42.196586 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:42.196909 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:42.196913 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:42.196914 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:42.197081 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:33:42.197189 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:42.197485 (+ 296us) server_negotiation.cc:299] Negotiation successful
0807 18:33:42.197632 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":51,"server-negotiator.queue_time_us":63}
I20260807 18:33:42.197805 20036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:42.189270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:45977 (local address 127.0.0.1:52808)
0807 18:33:42.189568 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:42.189582 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:42.189686 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:42.189910 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:42.189912 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:42.190306 (+ 394us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:33:42.190417 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:42.190422 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:42.191115 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:42.191117 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:42.191745 (+ 628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:42.191754 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:42.192206 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:42.192208 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:42.192270 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:42.192955 (+ 685us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:33:42.192980 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:33:42.194444 (+ 1464us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:33:42.196156 (+ 1712us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:33:42.196166 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:33:42.196173 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:33:42.196404 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:33:42.196701 (+ 297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:33:42.196704 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:33:42.196705 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:33:42.196797 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:33:42.197189 (+ 392us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:33:42.197197 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:33:42.197311 (+ 114us) client_negotiation.cc:769] Sending connection context
0807 18:33:42.197425 (+ 114us) client_negotiation.cc:240] Negotiation successful
0807 18:33:42.197579 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":120,"threads_started":1}
I20260807 18:33:42.201692 19464 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.11.14.126@KRBTEST.COM'} at 127.0.0.1:52808:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260807 18:33:42.202361 19464 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 18:33:42.207333 19610 tablet_service.cc:1467] Processing CreateTablet for tablet f7cbc15ec4c1478788a1e42afd9ae614 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=e84cf640c4fd48b88bbbe4970705e0f2]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 18:33:42.207393 19754 tablet_service.cc:1467] Processing CreateTablet for tablet f7cbc15ec4c1478788a1e42afd9ae614 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=e84cf640c4fd48b88bbbe4970705e0f2]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 18:33:42.207456 19902 tablet_service.cc:1467] Processing CreateTablet for tablet f7cbc15ec4c1478788a1e42afd9ae614 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=e84cf640c4fd48b88bbbe4970705e0f2]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 18:33:42.207787 19754 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f7cbc15ec4c1478788a1e42afd9ae614. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:33:42.207787 19610 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f7cbc15ec4c1478788a1e42afd9ae614. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:33:42.207896 19902 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f7cbc15ec4c1478788a1e42afd9ae614. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:33:42.210282 20000 tablet_bootstrap.cc:492] T f7cbc15ec4c1478788a1e42afd9ae614 P 841d14d044a3450dba7a52ace1d8f9f9: Bootstrap starting.
I20260807 18:33:42.210767 20002 tablet_bootstrap.cc:492] T f7cbc15ec4c1478788a1e42afd9ae614 P b8b91f4589f343c88246ca60212c015a: Bootstrap starting.
I20260807 18:33:42.211288 20000 tablet_bootstrap.cc:654] T f7cbc15ec4c1478788a1e42afd9ae614 P 841d14d044a3450dba7a52ace1d8f9f9: Neither blocks nor log segments found. Creating new log.
I20260807 18:33:42.211398 20001 tablet_bootstrap.cc:492] T f7cbc15ec4c1478788a1e42afd9ae614 P 973fa32d73684f51949933c03a32ce8e: Bootstrap starting.
I20260807 18:33:42.211824 20002 tablet_bootstrap.cc:654] T f7cbc15ec4c1478788a1e42afd9ae614 P b8b91f4589f343c88246ca60212c015a: Neither blocks nor log segments found. Creating new log.
I20260807 18:33:42.212370 20001 tablet_bootstrap.cc:654] T f7cbc15ec4c1478788a1e42afd9ae614 P 973fa32d73684f51949933c03a32ce8e: Neither blocks nor log segments found. Creating new log.
I20260807 18:33:42.212527 20000 tablet_bootstrap.cc:492] T f7cbc15ec4c1478788a1e42afd9ae614 P 841d14d044a3450dba7a52ace1d8f9f9: No bootstrap required, opened a new log
I20260807 18:33:42.212585 20000 ts_tablet_manager.cc:1397] T f7cbc15ec4c1478788a1e42afd9ae614 P 841d14d044a3450dba7a52ace1d8f9f9: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260807 18:33:42.213058 20002 tablet_bootstrap.cc:492] T f7cbc15ec4c1478788a1e42afd9ae614 P b8b91f4589f343c88246ca60212c015a: No bootstrap required, opened a new log
I20260807 18:33:42.213024 20000 raft_consensus.cc:348] T f7cbc15ec4c1478788a1e42afd9ae614 P 841d14d044a3450dba7a52ace1d8f9f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8b91f4589f343c88246ca60212c015a" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 41543 } } peers { permanent_uuid: "973fa32d73684f51949933c03a32ce8e" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 43313 } } peers { permanent_uuid: "841d14d044a3450dba7a52ace1d8f9f9" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 35051 } }
I20260807 18:33:42.213136 20002 ts_tablet_manager.cc:1397] T f7cbc15ec4c1478788a1e42afd9ae614 P b8b91f4589f343c88246ca60212c015a: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260807 18:33:42.213136 20000 raft_consensus.cc:374] T f7cbc15ec4c1478788a1e42afd9ae614 P 841d14d044a3450dba7a52ace1d8f9f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:33:42.213171 20000 raft_consensus.cc:729] T f7cbc15ec4c1478788a1e42afd9ae614 P 841d14d044a3450dba7a52ace1d8f9f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 841d14d044a3450dba7a52ace1d8f9f9, State: Initialized, Role: FOLLOWER
I20260807 18:33:42.213297 20000 consensus_queue.cc:260] T f7cbc15ec4c1478788a1e42afd9ae614 P 841d14d044a3450dba7a52ace1d8f9f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8b91f4589f343c88246ca60212c015a" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 41543 } } peers { permanent_uuid: "973fa32d73684f51949933c03a32ce8e" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 43313 } } peers { permanent_uuid: "841d14d044a3450dba7a52ace1d8f9f9" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 35051 } }
I20260807 18:33:42.213587 20000 ts_tablet_manager.cc:1428] T f7cbc15ec4c1478788a1e42afd9ae614 P 841d14d044a3450dba7a52ace1d8f9f9: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260807 18:33:42.213567 20010 tablet_replica.cc:439] T f7cbc15ec4c1478788a1e42afd9ae614 P 841d14d044a3450dba7a52ace1d8f9f9: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8b91f4589f343c88246ca60212c015a" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 41543 } } peers { permanent_uuid: "973fa32d73684f51949933c03a32ce8e" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 43313 } } peers { permanent_uuid: "841d14d044a3450dba7a52ace1d8f9f9" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 35051 } } }
I20260807 18:33:42.213590 20002 raft_consensus.cc:348] T f7cbc15ec4c1478788a1e42afd9ae614 P b8b91f4589f343c88246ca60212c015a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8b91f4589f343c88246ca60212c015a" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 41543 } } peers { permanent_uuid: "973fa32d73684f51949933c03a32ce8e" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 43313 } } peers { permanent_uuid: "841d14d044a3450dba7a52ace1d8f9f9" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 35051 } }
I20260807 18:33:42.213670 20001 tablet_bootstrap.cc:492] T f7cbc15ec4c1478788a1e42afd9ae614 P 973fa32d73684f51949933c03a32ce8e: No bootstrap required, opened a new log
I20260807 18:33:42.213696 20002 raft_consensus.cc:374] T f7cbc15ec4c1478788a1e42afd9ae614 P b8b91f4589f343c88246ca60212c015a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:33:42.213706 20010 tablet_replica.cc:442] T f7cbc15ec4c1478788a1e42afd9ae614 P 841d14d044a3450dba7a52ace1d8f9f9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 18:33:42.213734 20002 raft_consensus.cc:729] T f7cbc15ec4c1478788a1e42afd9ae614 P b8b91f4589f343c88246ca60212c015a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8b91f4589f343c88246ca60212c015a, State: Initialized, Role: FOLLOWER
I20260807 18:33:42.213744 20001 ts_tablet_manager.cc:1397] T f7cbc15ec4c1478788a1e42afd9ae614 P 973fa32d73684f51949933c03a32ce8e: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260807 18:33:42.213883 20002 consensus_queue.cc:260] T f7cbc15ec4c1478788a1e42afd9ae614 P b8b91f4589f343c88246ca60212c015a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8b91f4589f343c88246ca60212c015a" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 41543 } } peers { permanent_uuid: "973fa32d73684f51949933c03a32ce8e" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 43313 } } peers { permanent_uuid: "841d14d044a3450dba7a52ace1d8f9f9" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 35051 } }
I20260807 18:33:42.214148 20002 ts_tablet_manager.cc:1428] T f7cbc15ec4c1478788a1e42afd9ae614 P b8b91f4589f343c88246ca60212c015a: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260807 18:33:42.214146 20008 tablet_replica.cc:439] T f7cbc15ec4c1478788a1e42afd9ae614 P b8b91f4589f343c88246ca60212c015a: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8b91f4589f343c88246ca60212c015a" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 41543 } } peers { permanent_uuid: "973fa32d73684f51949933c03a32ce8e" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 43313 } } peers { permanent_uuid: "841d14d044a3450dba7a52ace1d8f9f9" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 35051 } } }
I20260807 18:33:42.214218 20001 raft_consensus.cc:348] T f7cbc15ec4c1478788a1e42afd9ae614 P 973fa32d73684f51949933c03a32ce8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8b91f4589f343c88246ca60212c015a" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 41543 } } peers { permanent_uuid: "973fa32d73684f51949933c03a32ce8e" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 43313 } } peers { permanent_uuid: "841d14d044a3450dba7a52ace1d8f9f9" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 35051 } }
I20260807 18:33:42.214314 20001 raft_consensus.cc:374] T f7cbc15ec4c1478788a1e42afd9ae614 P 973fa32d73684f51949933c03a32ce8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:33:42.214327 20008 tablet_replica.cc:442] T f7cbc15ec4c1478788a1e42afd9ae614 P b8b91f4589f343c88246ca60212c015a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 18:33:42.214357 20001 raft_consensus.cc:729] T f7cbc15ec4c1478788a1e42afd9ae614 P 973fa32d73684f51949933c03a32ce8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 973fa32d73684f51949933c03a32ce8e, State: Initialized, Role: FOLLOWER
I20260807 18:33:42.214489 20001 consensus_queue.cc:260] T f7cbc15ec4c1478788a1e42afd9ae614 P 973fa32d73684f51949933c03a32ce8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8b91f4589f343c88246ca60212c015a" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 41543 } } peers { permanent_uuid: "973fa32d73684f51949933c03a32ce8e" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 43313 } } peers { permanent_uuid: "841d14d044a3450dba7a52ace1d8f9f9" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 35051 } }
I20260807 18:33:42.214794 20001 ts_tablet_manager.cc:1428] T f7cbc15ec4c1478788a1e42afd9ae614 P 973fa32d73684f51949933c03a32ce8e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 18:33:42.214766 20007 tablet_replica.cc:439] T f7cbc15ec4c1478788a1e42afd9ae614 P 973fa32d73684f51949933c03a32ce8e: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8b91f4589f343c88246ca60212c015a" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 41543 } } peers { permanent_uuid: "973fa32d73684f51949933c03a32ce8e" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 43313 } } peers { permanent_uuid: "841d14d044a3450dba7a52ace1d8f9f9" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 35051 } } }
I20260807 18:33:42.214915 20007 tablet_replica.cc:442] T f7cbc15ec4c1478788a1e42afd9ae614 P 973fa32d73684f51949933c03a32ce8e: This TxnStatusTablet replica's current role is: FOLLOWER
W20260807 18:33:42.223255 19972 tablet.cc:2369] T 49a0eda694ac4e358e65e146714b9d82 P 841d14d044a3450dba7a52ace1d8f9f9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:33:42.344976 20007 raft_consensus.cc:482] T f7cbc15ec4c1478788a1e42afd9ae614 P 973fa32d73684f51949933c03a32ce8e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:33:42.345136 20007 raft_consensus.cc:504] T f7cbc15ec4c1478788a1e42afd9ae614 P 973fa32d73684f51949933c03a32ce8e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8b91f4589f343c88246ca60212c015a" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 41543 } } peers { permanent_uuid: "973fa32d73684f51949933c03a32ce8e" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 43313 } } peers { permanent_uuid: "841d14d044a3450dba7a52ace1d8f9f9" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 35051 } }
I20260807 18:33:42.346433 20007 leader_election.cc:288] T f7cbc15ec4c1478788a1e42afd9ae614 P 973fa32d73684f51949933c03a32ce8e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8b91f4589f343c88246ca60212c015a (127.11.14.65:41543), 841d14d044a3450dba7a52ace1d8f9f9 (127.11.14.67:35051)
I20260807 18:33:42.350447 19999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:42.346800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.66:43181 (local address 127.11.14.65:41543)
0807 18:33:42.346986 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:42.346993 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:42.347084 (+ 91us) server_negotiation.cc:407] Connection header received
0807 18:33:42.347295 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:42.347298 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:42.347364 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:42.347469 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:33:42.347802 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:42.348325 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:42.349456 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:42.349866 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:42.350005 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:42.350063 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:42.350242 (+ 179us) server_negotiation.cc:299] Negotiation successful
0807 18:33:42.350329 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260807 18:33:42.350490 20041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:42.346654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.65:41543 (local address 127.11.14.66:43181)
0807 18:33:42.346976 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:42.346990 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:42.347143 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:42.347485 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:42.347488 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:42.347507 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:33:42.347678 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:42.347684 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:42.348445 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:42.348450 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:42.349321 (+ 871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:42.349331 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:42.349996 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:42.349999 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:42.350103 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:42.350129 (+ 26us) client_negotiation.cc:769] Sending connection context
0807 18:33:42.350224 (+ 95us) client_negotiation.cc:240] Negotiation successful
0807 18:33:42.350301 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":236,"mutex_wait_us":1,"spinlock_wait_cycles":78592,"thread_start_us":102,"threads_started":1}
I20260807 18:33:42.351110 20042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:42.347098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.67:35051 (local address 127.11.14.66:52763)
0807 18:33:42.347408 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:42.347423 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:42.347564 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:42.347878 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:42.347882 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:42.347901 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:33:42.348096 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:42.348109 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:42.349009 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:42.349012 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:42.349894 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:42.349904 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:42.350715 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:42.350720 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:42.350828 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:42.350854 (+ 26us) client_negotiation.cc:769] Sending connection context
0807 18:33:42.350947 (+ 93us) client_negotiation.cc:240] Negotiation successful
0807 18:33:42.350999 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":244,"spinlock_wait_cycles":896,"thread_start_us":105,"threads_started":1}
I20260807 18:33:42.351058 19630 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f7cbc15ec4c1478788a1e42afd9ae614" candidate_uuid: "973fa32d73684f51949933c03a32ce8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8b91f4589f343c88246ca60212c015a" is_pre_election: true
I20260807 18:33:42.351191 19996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:42.347224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.66:52763 (local address 127.11.14.67:35051)
0807 18:33:42.347388 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:42.347393 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:42.347506 (+ 113us) server_negotiation.cc:407] Connection header received
0807 18:33:42.347688 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:42.347691 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:42.347739 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:42.347830 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:33:42.348276 (+ 446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:42.348854 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:42.350041 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:42.350531 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:42.350686 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:42.350750 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:42.350973 (+ 223us) server_negotiation.cc:299] Negotiation successful
0807 18:33:42.351054 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260807 18:33:42.351254 19630 raft_consensus.cc:2454] T f7cbc15ec4c1478788a1e42afd9ae614 P b8b91f4589f343c88246ca60212c015a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 973fa32d73684f51949933c03a32ce8e in term 0.
I20260807 18:33:42.351703 19706 leader_election.cc:302] T f7cbc15ec4c1478788a1e42afd9ae614 P 973fa32d73684f51949933c03a32ce8e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 973fa32d73684f51949933c03a32ce8e, b8b91f4589f343c88246ca60212c015a; no voters:
I20260807 18:33:42.351735 19922 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f7cbc15ec4c1478788a1e42afd9ae614" candidate_uuid: "973fa32d73684f51949933c03a32ce8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "841d14d044a3450dba7a52ace1d8f9f9" is_pre_election: true
I20260807 18:33:42.352006 20007 raft_consensus.cc:2781] T f7cbc15ec4c1478788a1e42afd9ae614 P 973fa32d73684f51949933c03a32ce8e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:33:42.352021 19922 raft_consensus.cc:2454] T f7cbc15ec4c1478788a1e42afd9ae614 P 841d14d044a3450dba7a52ace1d8f9f9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 973fa32d73684f51949933c03a32ce8e in term 0.
I20260807 18:33:42.352090 20007 raft_consensus.cc:482] T f7cbc15ec4c1478788a1e42afd9ae614 P 973fa32d73684f51949933c03a32ce8e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:33:42.352133 20007 raft_consensus.cc:3037] T f7cbc15ec4c1478788a1e42afd9ae614 P 973fa32d73684f51949933c03a32ce8e [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:33:42.353216 20007 raft_consensus.cc:504] T f7cbc15ec4c1478788a1e42afd9ae614 P 973fa32d73684f51949933c03a32ce8e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8b91f4589f343c88246ca60212c015a" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 41543 } } peers { permanent_uuid: "973fa32d73684f51949933c03a32ce8e" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 43313 } } peers { permanent_uuid: "841d14d044a3450dba7a52ace1d8f9f9" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 35051 } }
I20260807 18:33:42.353514 20007 leader_election.cc:288] T f7cbc15ec4c1478788a1e42afd9ae614 P 973fa32d73684f51949933c03a32ce8e [CANDIDATE]: Term 1 election: Requested vote from peers b8b91f4589f343c88246ca60212c015a (127.11.14.65:41543), 841d14d044a3450dba7a52ace1d8f9f9 (127.11.14.67:35051)
I20260807 18:33:42.353986 19922 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f7cbc15ec4c1478788a1e42afd9ae614" candidate_uuid: "973fa32d73684f51949933c03a32ce8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "841d14d044a3450dba7a52ace1d8f9f9"
I20260807 18:33:42.353984 19630 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f7cbc15ec4c1478788a1e42afd9ae614" candidate_uuid: "973fa32d73684f51949933c03a32ce8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8b91f4589f343c88246ca60212c015a"
I20260807 18:33:42.354115 19922 raft_consensus.cc:3037] T f7cbc15ec4c1478788a1e42afd9ae614 P 841d14d044a3450dba7a52ace1d8f9f9 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:33:42.354130 19630 raft_consensus.cc:3037] T f7cbc15ec4c1478788a1e42afd9ae614 P b8b91f4589f343c88246ca60212c015a [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:33:42.355084 19922 raft_consensus.cc:2454] T f7cbc15ec4c1478788a1e42afd9ae614 P 841d14d044a3450dba7a52ace1d8f9f9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 973fa32d73684f51949933c03a32ce8e in term 1.
I20260807 18:33:42.355182 19630 raft_consensus.cc:2454] T f7cbc15ec4c1478788a1e42afd9ae614 P b8b91f4589f343c88246ca60212c015a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 973fa32d73684f51949933c03a32ce8e in term 1.
I20260807 18:33:42.355423 19706 leader_election.cc:302] T f7cbc15ec4c1478788a1e42afd9ae614 P 973fa32d73684f51949933c03a32ce8e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 841d14d044a3450dba7a52ace1d8f9f9, 973fa32d73684f51949933c03a32ce8e; no voters:
I20260807 18:33:42.355655 20007 raft_consensus.cc:2781] T f7cbc15ec4c1478788a1e42afd9ae614 P 973fa32d73684f51949933c03a32ce8e [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:33:42.355875 20007 raft_consensus.cc:686] T f7cbc15ec4c1478788a1e42afd9ae614 P 973fa32d73684f51949933c03a32ce8e [term 1 LEADER]: Becoming Leader. State: Replica: 973fa32d73684f51949933c03a32ce8e, State: Running, Role: LEADER
I20260807 18:33:42.356195 20007 consensus_queue.cc:237] T f7cbc15ec4c1478788a1e42afd9ae614 P 973fa32d73684f51949933c03a32ce8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8b91f4589f343c88246ca60212c015a" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 41543 } } peers { permanent_uuid: "973fa32d73684f51949933c03a32ce8e" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 43313 } } peers { permanent_uuid: "841d14d044a3450dba7a52ace1d8f9f9" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 35051 } }
I20260807 18:33:42.357514 20043 tablet_replica.cc:439] T f7cbc15ec4c1478788a1e42afd9ae614 P 973fa32d73684f51949933c03a32ce8e: TxnStatusTablet state changed. Reason: New leader 973fa32d73684f51949933c03a32ce8e. Latest consensus state: current_term: 1 leader_uuid: "973fa32d73684f51949933c03a32ce8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8b91f4589f343c88246ca60212c015a" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 41543 } } peers { permanent_uuid: "973fa32d73684f51949933c03a32ce8e" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 43313 } } peers { permanent_uuid: "841d14d044a3450dba7a52ace1d8f9f9" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 35051 } } }
I20260807 18:33:42.357642 20043 tablet_replica.cc:442] T f7cbc15ec4c1478788a1e42afd9ae614 P 973fa32d73684f51949933c03a32ce8e: This TxnStatusTablet replica's current role is: LEADER
I20260807 18:33:42.357936 20045 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260807 18:33:42.358105 20045 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260807 18:33:42.358896 19462 catalog_manager.cc:5495] T f7cbc15ec4c1478788a1e42afd9ae614 P 973fa32d73684f51949933c03a32ce8e reported cstate change: term changed from 0 to 1, leader changed from <none> to 973fa32d73684f51949933c03a32ce8e (127.11.14.66). New cstate: current_term: 1 leader_uuid: "973fa32d73684f51949933c03a32ce8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8b91f4589f343c88246ca60212c015a" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 41543 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "973fa32d73684f51949933c03a32ce8e" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 43313 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "841d14d044a3450dba7a52ace1d8f9f9" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 35051 } health_report { overall_health: UNKNOWN } } }
I20260807 18:33:42.372597 19997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:42.368541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41820 (local address 127.11.14.66:43313)
0807 18:33:42.368662 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:42.368666 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:42.368679 (+ 13us) server_negotiation.cc:407] Connection header received
0807 18:33:42.368850 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:42.368852 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:42.368905 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:42.368992 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:33:42.369294 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:42.369829 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:42.371090 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:42.371299 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:42.371392 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:42.371743 (+ 351us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:33:42.372139 (+ 396us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:33:42.372229 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:42.372402 (+ 173us) server_negotiation.cc:299] Negotiation successful
0807 18:33:42.372461 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260807 18:33:42.372651 20036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:42.368454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.66:43313 (local address 127.0.0.1:41820)
0807 18:33:42.368599 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:42.368617 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:42.368717 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:42.369039 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:42.369042 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:42.369052 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 18:33:42.369167 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:42.369174 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:42.369941 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:42.369945 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:42.370974 (+ 1029us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:42.370987 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:42.371430 (+ 443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:42.371434 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:42.371524 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:42.371603 (+ 79us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 18:33:42.372242 (+ 639us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 18:33:42.372265 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 18:33:42.372354 (+ 89us) client_negotiation.cc:240] Negotiation successful
0807 18:33:42.372442 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260807 18:33:42.376384 19630 raft_consensus.cc:1261] T f7cbc15ec4c1478788a1e42afd9ae614 P b8b91f4589f343c88246ca60212c015a [term 1 FOLLOWER]: Refusing update from remote peer 973fa32d73684f51949933c03a32ce8e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:33:42.376643 19922 raft_consensus.cc:1261] T f7cbc15ec4c1478788a1e42afd9ae614 P 841d14d044a3450dba7a52ace1d8f9f9 [term 1 FOLLOWER]: Refusing update from remote peer 973fa32d73684f51949933c03a32ce8e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:33:42.376988 20007 consensus_queue.cc:1035] T f7cbc15ec4c1478788a1e42afd9ae614 P 973fa32d73684f51949933c03a32ce8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8b91f4589f343c88246ca60212c015a" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 41543 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:33:42.377188 20043 consensus_queue.cc:1035] T f7cbc15ec4c1478788a1e42afd9ae614 P 973fa32d73684f51949933c03a32ce8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "841d14d044a3450dba7a52ace1d8f9f9" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 35051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:33:42.378762 20008 tablet_replica.cc:439] T f7cbc15ec4c1478788a1e42afd9ae614 P b8b91f4589f343c88246ca60212c015a: TxnStatusTablet state changed. Reason: New leader 973fa32d73684f51949933c03a32ce8e. Latest consensus state: current_term: 1 leader_uuid: "973fa32d73684f51949933c03a32ce8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8b91f4589f343c88246ca60212c015a" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 41543 } } peers { permanent_uuid: "973fa32d73684f51949933c03a32ce8e" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 43313 } } peers { permanent_uuid: "841d14d044a3450dba7a52ace1d8f9f9" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 35051 } } }
I20260807 18:33:42.378881 20008 tablet_replica.cc:442] T f7cbc15ec4c1478788a1e42afd9ae614 P b8b91f4589f343c88246ca60212c015a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 18:33:42.378931 20010 tablet_replica.cc:439] T f7cbc15ec4c1478788a1e42afd9ae614 P 841d14d044a3450dba7a52ace1d8f9f9: TxnStatusTablet state changed. Reason: New leader 973fa32d73684f51949933c03a32ce8e. Latest consensus state: current_term: 1 leader_uuid: "973fa32d73684f51949933c03a32ce8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8b91f4589f343c88246ca60212c015a" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 41543 } } peers { permanent_uuid: "973fa32d73684f51949933c03a32ce8e" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 43313 } } peers { permanent_uuid: "841d14d044a3450dba7a52ace1d8f9f9" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 35051 } } }
I20260807 18:33:42.379060 20010 tablet_replica.cc:442] T f7cbc15ec4c1478788a1e42afd9ae614 P 841d14d044a3450dba7a52ace1d8f9f9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 18:33:42.381721 20007 tablet_replica.cc:439] T f7cbc15ec4c1478788a1e42afd9ae614 P 973fa32d73684f51949933c03a32ce8e: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "973fa32d73684f51949933c03a32ce8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8b91f4589f343c88246ca60212c015a" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 41543 } } peers { permanent_uuid: "973fa32d73684f51949933c03a32ce8e" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 43313 } } peers { permanent_uuid: "841d14d044a3450dba7a52ace1d8f9f9" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 35051 } } }
I20260807 18:33:42.381767 20043 tablet_replica.cc:439] T f7cbc15ec4c1478788a1e42afd9ae614 P 973fa32d73684f51949933c03a32ce8e: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "973fa32d73684f51949933c03a32ce8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8b91f4589f343c88246ca60212c015a" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 41543 } } peers { permanent_uuid: "973fa32d73684f51949933c03a32ce8e" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 43313 } } peers { permanent_uuid: "841d14d044a3450dba7a52ace1d8f9f9" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 35051 } } }
I20260807 18:33:42.381891 20043 tablet_replica.cc:442] T f7cbc15ec4c1478788a1e42afd9ae614 P 973fa32d73684f51949933c03a32ce8e: This TxnStatusTablet replica's current role is: LEADER
I20260807 18:33:42.381925 20010 tablet_replica.cc:439] T f7cbc15ec4c1478788a1e42afd9ae614 P 841d14d044a3450dba7a52ace1d8f9f9: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "973fa32d73684f51949933c03a32ce8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8b91f4589f343c88246ca60212c015a" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 41543 } } peers { permanent_uuid: "973fa32d73684f51949933c03a32ce8e" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 43313 } } peers { permanent_uuid: "841d14d044a3450dba7a52ace1d8f9f9" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 35051 } } }
I20260807 18:33:42.382020 20010 tablet_replica.cc:442] T f7cbc15ec4c1478788a1e42afd9ae614 P 841d14d044a3450dba7a52ace1d8f9f9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 18:33:42.383519 20008 tablet_replica.cc:439] T f7cbc15ec4c1478788a1e42afd9ae614 P b8b91f4589f343c88246ca60212c015a: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "973fa32d73684f51949933c03a32ce8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8b91f4589f343c88246ca60212c015a" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 41543 } } peers { permanent_uuid: "973fa32d73684f51949933c03a32ce8e" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 43313 } } peers { permanent_uuid: "841d14d044a3450dba7a52ace1d8f9f9" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 35051 } } }
I20260807 18:33:42.383665 20008 tablet_replica.cc:442] T f7cbc15ec4c1478788a1e42afd9ae614 P b8b91f4589f343c88246ca60212c015a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 18:33:42.381844 20007 tablet_replica.cc:442] T f7cbc15ec4c1478788a1e42afd9ae614 P 973fa32d73684f51949933c03a32ce8e: This TxnStatusTablet replica's current role is: LEADER
I20260807 18:33:42.396767 19997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:42.393123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.67:57903 (local address 127.11.14.66:43313)
0807 18:33:42.393247 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:42.393253 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:42.393538 (+ 285us) server_negotiation.cc:407] Connection header received
0807 18:33:42.393699 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:42.393702 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:42.393756 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:42.393855 (+ 99us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:33:42.394183 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:42.394685 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:42.395621 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:42.395846 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:42.395957 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:42.396270 (+ 313us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:33:42.396345 (+ 75us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:33:42.396426 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:42.396568 (+ 142us) server_negotiation.cc:299] Negotiation successful
0807 18:33:42.396620 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260807 18:33:42.396767 20054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:42.393024 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.66:43313 (local address 127.11.14.67:57903)
0807 18:33:42.393434 (+ 410us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:42.393464 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:42.393595 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:42.393906 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:42.393909 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:42.393918 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 18:33:42.394070 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:42.394075 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:42.394797 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:42.394801 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:42.395443 (+ 642us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:42.395451 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:42.395959 (+ 508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:42.395963 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:42.396055 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:42.396101 (+ 46us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 18:33:42.396442 (+ 341us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 18:33:42.396449 (+ 7us) client_negotiation.cc:769] Sending connection context
0807 18:33:42.396544 (+ 95us) client_negotiation.cc:240] Negotiation successful
0807 18:33:42.396630 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":299,"thread_start_us":110,"threads_started":1}
I20260807 18:33:42.403545 19630 raft_consensus.cc:1261] T 49a0eda694ac4e358e65e146714b9d82 P b8b91f4589f343c88246ca60212c015a [term 1 FOLLOWER]: Refusing update from remote peer 841d14d044a3450dba7a52ace1d8f9f9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:33:42.403553 19773 raft_consensus.cc:1261] T 49a0eda694ac4e358e65e146714b9d82 P 973fa32d73684f51949933c03a32ce8e [term 1 FOLLOWER]: Refusing update from remote peer 841d14d044a3450dba7a52ace1d8f9f9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:33:42.404079 20010 consensus_queue.cc:1035] T 49a0eda694ac4e358e65e146714b9d82 P 841d14d044a3450dba7a52ace1d8f9f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "973fa32d73684f51949933c03a32ce8e" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 43313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:33:42.404239 20006 consensus_queue.cc:1035] T 49a0eda694ac4e358e65e146714b9d82 P 841d14d044a3450dba7a52ace1d8f9f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8b91f4589f343c88246ca60212c015a" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 41543 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:33:42.427817 19996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:42.423810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.66:52625 (local address 127.11.14.67:35051)
0807 18:33:42.423944 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:42.423949 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:42.424169 (+ 220us) server_negotiation.cc:407] Connection header received
0807 18:33:42.424352 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:42.424357 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:42.424413 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:42.424492 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:33:42.424878 (+ 386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:42.425492 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:42.426735 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:42.426961 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:42.427045 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:42.427390 (+ 345us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:33:42.427458 (+ 68us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:33:42.427514 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:42.427650 (+ 136us) server_negotiation.cc:299] Negotiation successful
0807 18:33:42.427709 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260807 18:33:42.427886 20059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:42.423734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.67:35051 (local address 127.11.14.66:52625)
0807 18:33:42.424084 (+ 350us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:42.424096 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:42.424219 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:42.424607 (+ 388us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:42.424610 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:42.424618 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 18:33:42.424751 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:42.424757 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:42.425604 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:42.425608 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:42.426597 (+ 989us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:42.426608 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:42.427076 (+ 468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:42.427080 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:42.427187 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:42.427244 (+ 57us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 18:33:42.427544 (+ 300us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 18:33:42.427572 (+ 28us) client_negotiation.cc:769] Sending connection context
0807 18:33:42.427642 (+ 70us) client_negotiation.cc:240] Negotiation successful
0807 18:33:42.427725 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":136,"threads_started":1}
I20260807 18:33:42.453749 19463 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52800:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:33:42.453934 19463 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52800:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:33:42.455979 19463 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8482b9c6ca944b908d05b969396896b8: Sending DeleteTablet for 3 replicas of tablet 49a0eda694ac4e358e65e146714b9d82
I20260807 18:33:42.456614 19754 tablet_service.cc:1514] Processing DeleteTablet for tablet 49a0eda694ac4e358e65e146714b9d82 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:33:42 UTC) from {username='oryx', principal='oryx/127.11.14.126@KRBTEST.COM'} at 127.0.0.1:41810
I20260807 18:33:42.456746 19902 tablet_service.cc:1514] Processing DeleteTablet for tablet 49a0eda694ac4e358e65e146714b9d82 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:33:42 UTC) from {username='oryx', principal='oryx/127.11.14.126@KRBTEST.COM'} at 127.0.0.1:58600
I20260807 18:33:42.456768 19610 tablet_service.cc:1514] Processing DeleteTablet for tablet 49a0eda694ac4e358e65e146714b9d82 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:33:42 UTC) from {username='oryx', principal='oryx/127.11.14.126@KRBTEST.COM'} at 127.0.0.1:45022
I20260807 18:33:42.456815 20022 tablet_replica.cc:330] T 49a0eda694ac4e358e65e146714b9d82 P 973fa32d73684f51949933c03a32ce8e: stopping tablet replica
I20260807 18:33:42.456920 20024 tablet_replica.cc:330] T 49a0eda694ac4e358e65e146714b9d82 P 841d14d044a3450dba7a52ace1d8f9f9: stopping tablet replica
I20260807 18:33:42.456943 20025 tablet_replica.cc:330] T 49a0eda694ac4e358e65e146714b9d82 P b8b91f4589f343c88246ca60212c015a: stopping tablet replica
I20260807 18:33:42.456936 20022 raft_consensus.cc:2229] T 49a0eda694ac4e358e65e146714b9d82 P 973fa32d73684f51949933c03a32ce8e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:33:42.457031 20024 raft_consensus.cc:2229] T 49a0eda694ac4e358e65e146714b9d82 P 841d14d044a3450dba7a52ace1d8f9f9 [term 1 LEADER]: Raft consensus shutting down.
I20260807 18:33:42.457034 20025 raft_consensus.cc:2229] T 49a0eda694ac4e358e65e146714b9d82 P b8b91f4589f343c88246ca60212c015a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:33:42.457052 20022 raft_consensus.cc:2258] T 49a0eda694ac4e358e65e146714b9d82 P 973fa32d73684f51949933c03a32ce8e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:33:42.457141 20025 raft_consensus.cc:2258] T 49a0eda694ac4e358e65e146714b9d82 P b8b91f4589f343c88246ca60212c015a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:33:42.457199 20024 raft_consensus.cc:2258] T 49a0eda694ac4e358e65e146714b9d82 P 841d14d044a3450dba7a52ace1d8f9f9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:33:42.457952 20025 ts_tablet_manager.cc:1905] T 49a0eda694ac4e358e65e146714b9d82 P b8b91f4589f343c88246ca60212c015a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:33:42.458166 20024 ts_tablet_manager.cc:1905] T 49a0eda694ac4e358e65e146714b9d82 P 841d14d044a3450dba7a52ace1d8f9f9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:33:42.460807 20025 ts_tablet_manager.cc:1918] T 49a0eda694ac4e358e65e146714b9d82 P b8b91f4589f343c88246ca60212c015a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 18:33:42.460914 20025 log.cc:1196] T 49a0eda694ac4e358e65e146714b9d82 P b8b91f4589f343c88246ca60212c015a: Deleting WAL directory at /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127579900523-11321-0/minicluster-data/ts-0/wal/wals/49a0eda694ac4e358e65e146714b9d82
I20260807 18:33:42.461047 20024 ts_tablet_manager.cc:1918] T 49a0eda694ac4e358e65e146714b9d82 P 841d14d044a3450dba7a52ace1d8f9f9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 18:33:42.461130 20024 log.cc:1196] T 49a0eda694ac4e358e65e146714b9d82 P 841d14d044a3450dba7a52ace1d8f9f9: Deleting WAL directory at /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127579900523-11321-0/minicluster-data/ts-2/wal/wals/49a0eda694ac4e358e65e146714b9d82
I20260807 18:33:42.461226 20025 ts_tablet_manager.cc:1939] T 49a0eda694ac4e358e65e146714b9d82 P b8b91f4589f343c88246ca60212c015a: Deleting consensus metadata
I20260807 18:33:42.461408 20024 ts_tablet_manager.cc:1939] T 49a0eda694ac4e358e65e146714b9d82 P 841d14d044a3450dba7a52ace1d8f9f9: Deleting consensus metadata
I20260807 18:33:42.461825 20022 ts_tablet_manager.cc:1905] T 49a0eda694ac4e358e65e146714b9d82 P 973fa32d73684f51949933c03a32ce8e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:33:42.464370 20022 ts_tablet_manager.cc:1918] T 49a0eda694ac4e358e65e146714b9d82 P 973fa32d73684f51949933c03a32ce8e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 18:33:42.464468 20022 log.cc:1196] T 49a0eda694ac4e358e65e146714b9d82 P 973fa32d73684f51949933c03a32ce8e: Deleting WAL directory at /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127579900523-11321-0/minicluster-data/ts-1/wal/wals/49a0eda694ac4e358e65e146714b9d82
I20260807 18:33:42.464573 19451 catalog_manager.cc:4841] TS b8b91f4589f343c88246ca60212c015a (127.11.14.65:41543): tablet 49a0eda694ac4e358e65e146714b9d82 (table test-table [id=d4132ac9baa847dbb5cb305851a9fd70]) successfully deleted
I20260807 18:33:42.464737 20022 ts_tablet_manager.cc:1939] T 49a0eda694ac4e358e65e146714b9d82 P 973fa32d73684f51949933c03a32ce8e: Deleting consensus metadata
I20260807 18:33:42.465507 19450 catalog_manager.cc:4841] TS 973fa32d73684f51949933c03a32ce8e (127.11.14.66:43313): tablet 49a0eda694ac4e358e65e146714b9d82 (table test-table [id=d4132ac9baa847dbb5cb305851a9fd70]) successfully deleted
I20260807 18:33:42.466568 19451 catalog_manager.cc:4841] TS 841d14d044a3450dba7a52ace1d8f9f9 (127.11.14.67:35051): tablet 49a0eda694ac4e358e65e146714b9d82 (table test-table [id=d4132ac9baa847dbb5cb305851a9fd70]) successfully deleted
Aug 07 18:33:42 dist-test-slave-6k22 krb5kdc[19420](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127622, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 07 18:33:42 dist-test-slave-6k22 krb5kdc[19420](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.11.14.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:33:42 dist-test-slave-6k22 krb5kdc[19420](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.11.14.126@KRBTEST.COM, Server not found in Kerberos database
W20260807 18:33:42.490703 19517 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:33:42.484079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52820 (local address 127.11.14.126:45977)
0807 18:33:42.484232 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:42.484239 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:42.484458 (+ 219us) server_negotiation.cc:407] Connection header received
0807 18:33:42.484632 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:42.484635 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:42.484690 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:42.484779 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:33:42.485509 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:42.486115 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:42.487025 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:42.487257 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:42.487372 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:42.490578 (+ 3206us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52820: BlockingRecv error: recv got EOF from 127.0.0.1:52820 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Aug 07 18:33:42 dist-test-slave-6k22 krb5kdc[19420](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127622, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.11.14.126@KRBTEST.COM
I20260807 18:33:42.501281 19517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:42.492632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52826 (local address 127.11.14.126:45977)
0807 18:33:42.492755 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:42.492759 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:42.492961 (+ 202us) server_negotiation.cc:407] Connection header received
0807 18:33:42.493080 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:42.493083 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:42.493131 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:42.493213 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:33:42.493912 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:42.494412 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:42.495203 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:42.495419 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:42.495538 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:42.497991 (+ 2453us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:33:42.497995 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:33:42.497997 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:33:42.498020 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:33:42.499457 (+ 1437us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:42.499966 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:42.499970 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:42.499972 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:42.500055 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:42.500390 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:42.500393 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:42.500395 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:42.500572 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:33:42.500713 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:42.501003 (+ 290us) server_negotiation.cc:299] Negotiation successful
0807 18:33:42.501114 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260807 18:33:42.506520 19463 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52826:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 18:33:42.507004 19463 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 18:33:42.512158 19610 tablet_service.cc:1467] Processing CreateTablet for tablet cb0ef1d9d2c541648d248a0961e833cd (DEFAULT_TABLE table=test-table [id=9edeac37b02a454086affd1f8c2e5938]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:33:42.512367 19902 tablet_service.cc:1467] Processing CreateTablet for tablet cb0ef1d9d2c541648d248a0961e833cd (DEFAULT_TABLE table=test-table [id=9edeac37b02a454086affd1f8c2e5938]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:33:42.512519 19610 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cb0ef1d9d2c541648d248a0961e833cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:33:42.512677 19902 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cb0ef1d9d2c541648d248a0961e833cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:33:42.514829 20002 tablet_bootstrap.cc:492] T cb0ef1d9d2c541648d248a0961e833cd P b8b91f4589f343c88246ca60212c015a: Bootstrap starting.
I20260807 18:33:42.515342 20000 tablet_bootstrap.cc:492] T cb0ef1d9d2c541648d248a0961e833cd P 841d14d044a3450dba7a52ace1d8f9f9: Bootstrap starting.
I20260807 18:33:42.515278 19754 tablet_service.cc:1467] Processing CreateTablet for tablet cb0ef1d9d2c541648d248a0961e833cd (DEFAULT_TABLE table=test-table [id=9edeac37b02a454086affd1f8c2e5938]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:33:42.515640 19754 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cb0ef1d9d2c541648d248a0961e833cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:33:42.515853 20002 tablet_bootstrap.cc:654] T cb0ef1d9d2c541648d248a0961e833cd P b8b91f4589f343c88246ca60212c015a: Neither blocks nor log segments found. Creating new log.
I20260807 18:33:42.516301 20000 tablet_bootstrap.cc:654] T cb0ef1d9d2c541648d248a0961e833cd P 841d14d044a3450dba7a52ace1d8f9f9: Neither blocks nor log segments found. Creating new log.
I20260807 18:33:42.518095 20001 tablet_bootstrap.cc:492] T cb0ef1d9d2c541648d248a0961e833cd P 973fa32d73684f51949933c03a32ce8e: Bootstrap starting.
I20260807 18:33:42.519222 20001 tablet_bootstrap.cc:654] T cb0ef1d9d2c541648d248a0961e833cd P 973fa32d73684f51949933c03a32ce8e: Neither blocks nor log segments found. Creating new log.
I20260807 18:33:42.519506 20000 tablet_bootstrap.cc:492] T cb0ef1d9d2c541648d248a0961e833cd P 841d14d044a3450dba7a52ace1d8f9f9: No bootstrap required, opened a new log
I20260807 18:33:42.519626 20000 ts_tablet_manager.cc:1397] T cb0ef1d9d2c541648d248a0961e833cd P 841d14d044a3450dba7a52ace1d8f9f9: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260807 18:33:42.520087 20000 raft_consensus.cc:348] T cb0ef1d9d2c541648d248a0961e833cd P 841d14d044a3450dba7a52ace1d8f9f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8b91f4589f343c88246ca60212c015a" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 41543 } } peers { permanent_uuid: "973fa32d73684f51949933c03a32ce8e" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 43313 } } peers { permanent_uuid: "841d14d044a3450dba7a52ace1d8f9f9" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 35051 } }
I20260807 18:33:42.520220 20000 raft_consensus.cc:374] T cb0ef1d9d2c541648d248a0961e833cd P 841d14d044a3450dba7a52ace1d8f9f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:33:42.520258 20000 raft_consensus.cc:729] T cb0ef1d9d2c541648d248a0961e833cd P 841d14d044a3450dba7a52ace1d8f9f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 841d14d044a3450dba7a52ace1d8f9f9, State: Initialized, Role: FOLLOWER
I20260807 18:33:42.520403 20000 consensus_queue.cc:260] T cb0ef1d9d2c541648d248a0961e833cd P 841d14d044a3450dba7a52ace1d8f9f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8b91f4589f343c88246ca60212c015a" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 41543 } } peers { permanent_uuid: "973fa32d73684f51949933c03a32ce8e" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 43313 } } peers { permanent_uuid: "841d14d044a3450dba7a52ace1d8f9f9" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 35051 } }
I20260807 18:33:42.520694 20000 ts_tablet_manager.cc:1428] T cb0ef1d9d2c541648d248a0961e833cd P 841d14d044a3450dba7a52ace1d8f9f9: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 18:33:42.522182 20001 tablet_bootstrap.cc:492] T cb0ef1d9d2c541648d248a0961e833cd P 973fa32d73684f51949933c03a32ce8e: No bootstrap required, opened a new log
I20260807 18:33:42.522298 20001 ts_tablet_manager.cc:1397] T cb0ef1d9d2c541648d248a0961e833cd P 973fa32d73684f51949933c03a32ce8e: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.002s
I20260807 18:33:42.522810 20001 raft_consensus.cc:348] T cb0ef1d9d2c541648d248a0961e833cd P 973fa32d73684f51949933c03a32ce8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8b91f4589f343c88246ca60212c015a" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 41543 } } peers { permanent_uuid: "973fa32d73684f51949933c03a32ce8e" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 43313 } } peers { permanent_uuid: "841d14d044a3450dba7a52ace1d8f9f9" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 35051 } }
I20260807 18:33:42.522936 20001 raft_consensus.cc:374] T cb0ef1d9d2c541648d248a0961e833cd P 973fa32d73684f51949933c03a32ce8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:33:42.522979 20001 raft_consensus.cc:729] T cb0ef1d9d2c541648d248a0961e833cd P 973fa32d73684f51949933c03a32ce8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 973fa32d73684f51949933c03a32ce8e, State: Initialized, Role: FOLLOWER
I20260807 18:33:42.523123 20001 consensus_queue.cc:260] T cb0ef1d9d2c541648d248a0961e833cd P 973fa32d73684f51949933c03a32ce8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8b91f4589f343c88246ca60212c015a" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 41543 } } peers { permanent_uuid: "973fa32d73684f51949933c03a32ce8e" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 43313 } } peers { permanent_uuid: "841d14d044a3450dba7a52ace1d8f9f9" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 35051 } }
I20260807 18:33:42.523450 20001 ts_tablet_manager.cc:1428] T cb0ef1d9d2c541648d248a0961e833cd P 973fa32d73684f51949933c03a32ce8e: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260807 18:33:42.524503 20002 tablet_bootstrap.cc:492] T cb0ef1d9d2c541648d248a0961e833cd P b8b91f4589f343c88246ca60212c015a: No bootstrap required, opened a new log
I20260807 18:33:42.524608 20002 ts_tablet_manager.cc:1397] T cb0ef1d9d2c541648d248a0961e833cd P b8b91f4589f343c88246ca60212c015a: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.000s
I20260807 18:33:42.525104 20002 raft_consensus.cc:348] T cb0ef1d9d2c541648d248a0961e833cd P b8b91f4589f343c88246ca60212c015a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8b91f4589f343c88246ca60212c015a" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 41543 } } peers { permanent_uuid: "973fa32d73684f51949933c03a32ce8e" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 43313 } } peers { permanent_uuid: "841d14d044a3450dba7a52ace1d8f9f9" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 35051 } }
I20260807 18:33:42.525236 20002 raft_consensus.cc:374] T cb0ef1d9d2c541648d248a0961e833cd P b8b91f4589f343c88246ca60212c015a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:33:42.525278 20002 raft_consensus.cc:729] T cb0ef1d9d2c541648d248a0961e833cd P b8b91f4589f343c88246ca60212c015a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8b91f4589f343c88246ca60212c015a, State: Initialized, Role: FOLLOWER
I20260807 18:33:42.525441 20002 consensus_queue.cc:260] T cb0ef1d9d2c541648d248a0961e833cd P b8b91f4589f343c88246ca60212c015a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8b91f4589f343c88246ca60212c015a" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 41543 } } peers { permanent_uuid: "973fa32d73684f51949933c03a32ce8e" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 43313 } } peers { permanent_uuid: "841d14d044a3450dba7a52ace1d8f9f9" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 35051 } }
I20260807 18:33:42.525727 20002 ts_tablet_manager.cc:1428] T cb0ef1d9d2c541648d248a0961e833cd P b8b91f4589f343c88246ca60212c015a: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260807 18:33:42.556084 20008 raft_consensus.cc:482] T cb0ef1d9d2c541648d248a0961e833cd P b8b91f4589f343c88246ca60212c015a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:33:42.556221 20008 raft_consensus.cc:504] T cb0ef1d9d2c541648d248a0961e833cd P b8b91f4589f343c88246ca60212c015a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8b91f4589f343c88246ca60212c015a" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 41543 } } peers { permanent_uuid: "973fa32d73684f51949933c03a32ce8e" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 43313 } } peers { permanent_uuid: "841d14d044a3450dba7a52ace1d8f9f9" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 35051 } }
I20260807 18:33:42.557361 20008 leader_election.cc:288] T cb0ef1d9d2c541648d248a0961e833cd P b8b91f4589f343c88246ca60212c015a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 973fa32d73684f51949933c03a32ce8e (127.11.14.66:43313), 841d14d044a3450dba7a52ace1d8f9f9 (127.11.14.67:35051)
I20260807 18:33:42.561308 20078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:42.557605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.67:35051 (local address 127.11.14.65:58083)
0807 18:33:42.557900 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:42.557913 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:42.558040 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:42.558323 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:42.558326 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:42.558362 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:33:42.558512 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:42.558518 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:42.559297 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:42.559300 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:42.560185 (+ 885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:42.560194 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:42.560862 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:42.560866 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:42.560981 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:42.561000 (+ 19us) client_negotiation.cc:769] Sending connection context
0807 18:33:42.561090 (+ 90us) client_negotiation.cc:240] Negotiation successful
0807 18:33:42.561157 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":95,"threads_started":1}
I20260807 18:33:42.561311 19996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:42.557687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.65:58083 (local address 127.11.14.67:35051)
0807 18:33:42.557837 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:42.557843 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:42.557993 (+ 150us) server_negotiation.cc:407] Connection header received
0807 18:33:42.558163 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:42.558166 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:42.558222 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:42.558306 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:33:42.558636 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:42.559145 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:42.560320 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:42.560718 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:42.560853 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:42.560910 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:42.561109 (+ 199us) server_negotiation.cc:299] Negotiation successful
0807 18:33:42.561184 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260807 18:33:42.561651 20080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:42.557605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.66:43313 (local address 127.11.14.65:41545)
0807 18:33:42.558080 (+ 475us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:42.558091 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:42.558229 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:42.558526 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:42.558528 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:42.558546 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:33:42.558977 (+ 431us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:42.558984 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:42.559725 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:42.559730 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:42.560491 (+ 761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:42.560499 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:42.561170 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:42.561174 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:42.561301 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:42.561319 (+ 18us) client_negotiation.cc:769] Sending connection context
0807 18:33:42.561412 (+ 93us) client_negotiation.cc:240] Negotiation successful
0807 18:33:42.561503 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":420,"mutex_wait_us":32,"thread_start_us":101,"threads_started":1}
I20260807 18:33:42.561766 19922 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb0ef1d9d2c541648d248a0961e833cd" candidate_uuid: "b8b91f4589f343c88246ca60212c015a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "841d14d044a3450dba7a52ace1d8f9f9" is_pre_election: true
I20260807 18:33:42.561939 19922 raft_consensus.cc:2454] T cb0ef1d9d2c541648d248a0961e833cd P 841d14d044a3450dba7a52ace1d8f9f9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8b91f4589f343c88246ca60212c015a in term 0.
I20260807 18:33:42.561957 19997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:42.557684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.65:41545 (local address 127.11.14.66:43313)
0807 18:33:42.557958 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:42.557962 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:42.558189 (+ 227us) server_negotiation.cc:407] Connection header received
0807 18:33:42.558372 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:42.558378 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:42.558425 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:42.558512 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:33:42.559104 (+ 592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:42.559605 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:42.560626 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:42.561021 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:42.561169 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:42.561220 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:42.561716 (+ 496us) server_negotiation.cc:299] Negotiation successful
0807 18:33:42.561822 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":185}
I20260807 18:33:42.562418 19560 leader_election.cc:302] T cb0ef1d9d2c541648d248a0961e833cd P b8b91f4589f343c88246ca60212c015a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 841d14d044a3450dba7a52ace1d8f9f9, b8b91f4589f343c88246ca60212c015a; no voters:
I20260807 18:33:42.562428 19773 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb0ef1d9d2c541648d248a0961e833cd" candidate_uuid: "b8b91f4589f343c88246ca60212c015a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "973fa32d73684f51949933c03a32ce8e" is_pre_election: true
I20260807 18:33:42.562587 19773 raft_consensus.cc:2454] T cb0ef1d9d2c541648d248a0961e833cd P 973fa32d73684f51949933c03a32ce8e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8b91f4589f343c88246ca60212c015a in term 0.
I20260807 18:33:42.562674 20008 raft_consensus.cc:2781] T cb0ef1d9d2c541648d248a0961e833cd P b8b91f4589f343c88246ca60212c015a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:33:42.562741 20008 raft_consensus.cc:482] T cb0ef1d9d2c541648d248a0961e833cd P b8b91f4589f343c88246ca60212c015a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:33:42.562770 20008 raft_consensus.cc:3037] T cb0ef1d9d2c541648d248a0961e833cd P b8b91f4589f343c88246ca60212c015a [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:33:42.563989 20008 raft_consensus.cc:504] T cb0ef1d9d2c541648d248a0961e833cd P b8b91f4589f343c88246ca60212c015a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8b91f4589f343c88246ca60212c015a" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 41543 } } peers { permanent_uuid: "973fa32d73684f51949933c03a32ce8e" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 43313 } } peers { permanent_uuid: "841d14d044a3450dba7a52ace1d8f9f9" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 35051 } }
I20260807 18:33:42.564424 20008 leader_election.cc:288] T cb0ef1d9d2c541648d248a0961e833cd P b8b91f4589f343c88246ca60212c015a [CANDIDATE]: Term 1 election: Requested vote from peers 973fa32d73684f51949933c03a32ce8e (127.11.14.66:43313), 841d14d044a3450dba7a52ace1d8f9f9 (127.11.14.67:35051)
I20260807 18:33:42.564751 19773 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb0ef1d9d2c541648d248a0961e833cd" candidate_uuid: "b8b91f4589f343c88246ca60212c015a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "973fa32d73684f51949933c03a32ce8e"
I20260807 18:33:42.564795 19922 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb0ef1d9d2c541648d248a0961e833cd" candidate_uuid: "b8b91f4589f343c88246ca60212c015a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "841d14d044a3450dba7a52ace1d8f9f9"
I20260807 18:33:42.564914 19922 raft_consensus.cc:3037] T cb0ef1d9d2c541648d248a0961e833cd P 841d14d044a3450dba7a52ace1d8f9f9 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:33:42.564915 19773 raft_consensus.cc:3037] T cb0ef1d9d2c541648d248a0961e833cd P 973fa32d73684f51949933c03a32ce8e [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:33:42.566062 19922 raft_consensus.cc:2454] T cb0ef1d9d2c541648d248a0961e833cd P 841d14d044a3450dba7a52ace1d8f9f9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8b91f4589f343c88246ca60212c015a in term 1.
I20260807 18:33:42.566062 19773 raft_consensus.cc:2454] T cb0ef1d9d2c541648d248a0961e833cd P 973fa32d73684f51949933c03a32ce8e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8b91f4589f343c88246ca60212c015a in term 1.
I20260807 18:33:42.566460 19559 leader_election.cc:302] T cb0ef1d9d2c541648d248a0961e833cd P b8b91f4589f343c88246ca60212c015a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 973fa32d73684f51949933c03a32ce8e, b8b91f4589f343c88246ca60212c015a; no voters:
I20260807 18:33:42.566660 20008 raft_consensus.cc:2781] T cb0ef1d9d2c541648d248a0961e833cd P b8b91f4589f343c88246ca60212c015a [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:33:42.566918 20008 raft_consensus.cc:686] T cb0ef1d9d2c541648d248a0961e833cd P b8b91f4589f343c88246ca60212c015a [term 1 LEADER]: Becoming Leader. State: Replica: b8b91f4589f343c88246ca60212c015a, State: Running, Role: LEADER
I20260807 18:33:42.567178 20008 consensus_queue.cc:237] T cb0ef1d9d2c541648d248a0961e833cd P b8b91f4589f343c88246ca60212c015a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8b91f4589f343c88246ca60212c015a" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 41543 } } peers { permanent_uuid: "973fa32d73684f51949933c03a32ce8e" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 43313 } } peers { permanent_uuid: "841d14d044a3450dba7a52ace1d8f9f9" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 35051 } }
I20260807 18:33:42.569613 19463 catalog_manager.cc:5495] T cb0ef1d9d2c541648d248a0961e833cd P b8b91f4589f343c88246ca60212c015a reported cstate change: term changed from 0 to 1, leader changed from <none> to b8b91f4589f343c88246ca60212c015a (127.11.14.65). New cstate: current_term: 1 leader_uuid: "b8b91f4589f343c88246ca60212c015a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8b91f4589f343c88246ca60212c015a" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 41543 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "973fa32d73684f51949933c03a32ce8e" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 43313 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "841d14d044a3450dba7a52ace1d8f9f9" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 35051 } health_report { overall_health: UNKNOWN } } }
I20260807 18:33:42.586025 19999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:42.582298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45024 (local address 127.11.14.65:41543)
0807 18:33:42.582431 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:42.582437 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:42.582453 (+ 16us) server_negotiation.cc:407] Connection header received
0807 18:33:42.582652 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:42.582654 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:42.582709 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:42.582827 (+ 118us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:33:42.583077 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:42.583652 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:42.584508 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:42.584733 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:42.584849 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:42.585140 (+ 291us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:33:42.585619 (+ 479us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:33:42.585750 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:42.585833 (+ 83us) server_negotiation.cc:299] Negotiation successful
0807 18:33:42.585887 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260807 18:33:42.588668 19922 raft_consensus.cc:1261] T cb0ef1d9d2c541648d248a0961e833cd P 841d14d044a3450dba7a52ace1d8f9f9 [term 1 FOLLOWER]: Refusing update from remote peer b8b91f4589f343c88246ca60212c015a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:33:42.588675 19773 raft_consensus.cc:1261] T cb0ef1d9d2c541648d248a0961e833cd P 973fa32d73684f51949933c03a32ce8e [term 1 FOLLOWER]: Refusing update from remote peer b8b91f4589f343c88246ca60212c015a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:33:42.589229 20008 consensus_queue.cc:1035] T cb0ef1d9d2c541648d248a0961e833cd P b8b91f4589f343c88246ca60212c015a [LEADER]: Connected to new peer: Peer: permanent_uuid: "841d14d044a3450dba7a52ace1d8f9f9" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 35051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:33:42.589439 20081 consensus_queue.cc:1035] T cb0ef1d9d2c541648d248a0961e833cd P b8b91f4589f343c88246ca60212c015a [LEADER]: Connected to new peer: Peer: permanent_uuid: "973fa32d73684f51949933c03a32ce8e" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 43313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:33:42.600745 19465 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52826:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:33:42.600871 19465 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52826:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:33:42.602862 19465 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8482b9c6ca944b908d05b969396896b8: Sending DeleteTablet for 3 replicas of tablet cb0ef1d9d2c541648d248a0961e833cd
I20260807 18:33:42.603492 19610 tablet_service.cc:1514] Processing DeleteTablet for tablet cb0ef1d9d2c541648d248a0961e833cd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:33:42 UTC) from {username='oryx', principal='oryx/127.11.14.126@KRBTEST.COM'} at 127.0.0.1:45022
I20260807 18:33:42.603495 19902 tablet_service.cc:1514] Processing DeleteTablet for tablet cb0ef1d9d2c541648d248a0961e833cd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:33:42 UTC) from {username='oryx', principal='oryx/127.11.14.126@KRBTEST.COM'} at 127.0.0.1:58600
I20260807 18:33:42.603695 20024 tablet_replica.cc:330] T cb0ef1d9d2c541648d248a0961e833cd P 841d14d044a3450dba7a52ace1d8f9f9: stopping tablet replica
I20260807 18:33:42.603693 20025 tablet_replica.cc:330] T cb0ef1d9d2c541648d248a0961e833cd P b8b91f4589f343c88246ca60212c015a: stopping tablet replica
I20260807 18:33:42.603679 19754 tablet_service.cc:1514] Processing DeleteTablet for tablet cb0ef1d9d2c541648d248a0961e833cd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:33:42 UTC) from {username='oryx', principal='oryx/127.11.14.126@KRBTEST.COM'} at 127.0.0.1:41810
I20260807 18:33:42.603820 20025 raft_consensus.cc:2229] T cb0ef1d9d2c541648d248a0961e833cd P b8b91f4589f343c88246ca60212c015a [term 1 LEADER]: Raft consensus shutting down.
I20260807 18:33:42.603826 20024 raft_consensus.cc:2229] T cb0ef1d9d2c541648d248a0961e833cd P 841d14d044a3450dba7a52ace1d8f9f9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:33:42.603850 20022 tablet_replica.cc:330] T cb0ef1d9d2c541648d248a0961e833cd P 973fa32d73684f51949933c03a32ce8e: stopping tablet replica
I20260807 18:33:42.603951 20022 raft_consensus.cc:2229] T cb0ef1d9d2c541648d248a0961e833cd P 973fa32d73684f51949933c03a32ce8e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:33:42.603957 20024 raft_consensus.cc:2258] T cb0ef1d9d2c541648d248a0961e833cd P 841d14d044a3450dba7a52ace1d8f9f9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:33:42.604067 20022 raft_consensus.cc:2258] T cb0ef1d9d2c541648d248a0961e833cd P 973fa32d73684f51949933c03a32ce8e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:33:42.604100 20025 raft_consensus.cc:2258] T cb0ef1d9d2c541648d248a0961e833cd P b8b91f4589f343c88246ca60212c015a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:33:42.604769 20024 ts_tablet_manager.cc:1905] T cb0ef1d9d2c541648d248a0961e833cd P 841d14d044a3450dba7a52ace1d8f9f9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:33:42.604794 20022 ts_tablet_manager.cc:1905] T cb0ef1d9d2c541648d248a0961e833cd P 973fa32d73684f51949933c03a32ce8e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:33:42.604900 20025 ts_tablet_manager.cc:1905] T cb0ef1d9d2c541648d248a0961e833cd P b8b91f4589f343c88246ca60212c015a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:33:42.607226 20024 ts_tablet_manager.cc:1918] T cb0ef1d9d2c541648d248a0961e833cd P 841d14d044a3450dba7a52ace1d8f9f9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:33:42.607226 20022 ts_tablet_manager.cc:1918] T cb0ef1d9d2c541648d248a0961e833cd P 973fa32d73684f51949933c03a32ce8e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:33:42.607298 20022 log.cc:1196] T cb0ef1d9d2c541648d248a0961e833cd P 973fa32d73684f51949933c03a32ce8e: Deleting WAL directory at /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127579900523-11321-0/minicluster-data/ts-1/wal/wals/cb0ef1d9d2c541648d248a0961e833cd
I20260807 18:33:42.607298 20024 log.cc:1196] T cb0ef1d9d2c541648d248a0961e833cd P 841d14d044a3450dba7a52ace1d8f9f9: Deleting WAL directory at /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127579900523-11321-0/minicluster-data/ts-2/wal/wals/cb0ef1d9d2c541648d248a0961e833cd
I20260807 18:33:42.607471 20025 ts_tablet_manager.cc:1918] T cb0ef1d9d2c541648d248a0961e833cd P b8b91f4589f343c88246ca60212c015a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:33:42.607595 20024 ts_tablet_manager.cc:1939] T cb0ef1d9d2c541648d248a0961e833cd P 841d14d044a3450dba7a52ace1d8f9f9: Deleting consensus metadata
I20260807 18:33:42.607545 20025 log.cc:1196] T cb0ef1d9d2c541648d248a0961e833cd P b8b91f4589f343c88246ca60212c015a: Deleting WAL directory at /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127579900523-11321-0/minicluster-data/ts-0/wal/wals/cb0ef1d9d2c541648d248a0961e833cd
I20260807 18:33:42.608119 20025 ts_tablet_manager.cc:1939] T cb0ef1d9d2c541648d248a0961e833cd P b8b91f4589f343c88246ca60212c015a: Deleting consensus metadata
I20260807 18:33:42.608268 19451 catalog_manager.cc:4841] TS 841d14d044a3450dba7a52ace1d8f9f9 (127.11.14.67:35051): tablet cb0ef1d9d2c541648d248a0961e833cd (table test-table [id=9edeac37b02a454086affd1f8c2e5938]) successfully deleted
I20260807 18:33:42.608376 19465 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52826:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 18:33:42.608846 19465 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 18:33:42.609079 19451 catalog_manager.cc:4841] TS b8b91f4589f343c88246ca60212c015a (127.11.14.65:41543): tablet cb0ef1d9d2c541648d248a0961e833cd (table test-table [id=9edeac37b02a454086affd1f8c2e5938]) successfully deleted
I20260807 18:33:42.609548 20022 ts_tablet_manager.cc:1939] T cb0ef1d9d2c541648d248a0961e833cd P 973fa32d73684f51949933c03a32ce8e: Deleting consensus metadata
I20260807 18:33:42.610190 19450 catalog_manager.cc:4841] TS 973fa32d73684f51949933c03a32ce8e (127.11.14.66:43313): tablet cb0ef1d9d2c541648d248a0961e833cd (table test-table [id=9edeac37b02a454086affd1f8c2e5938]) successfully deleted
I20260807 18:33:42.613564 19610 tablet_service.cc:1467] Processing CreateTablet for tablet 35a1d137abb34ed39bb537071065ed1b (DEFAULT_TABLE table=test-table [id=2eb5c58169cd42f8b71b4c3ef26391d1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:33:42.613709 19902 tablet_service.cc:1467] Processing CreateTablet for tablet 35a1d137abb34ed39bb537071065ed1b (DEFAULT_TABLE table=test-table [id=2eb5c58169cd42f8b71b4c3ef26391d1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:33:42.613819 19610 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 35a1d137abb34ed39bb537071065ed1b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:33:42.613937 19902 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 35a1d137abb34ed39bb537071065ed1b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:33:42.616178 20002 tablet_bootstrap.cc:492] T 35a1d137abb34ed39bb537071065ed1b P b8b91f4589f343c88246ca60212c015a: Bootstrap starting.
I20260807 18:33:42.616211 20000 tablet_bootstrap.cc:492] T 35a1d137abb34ed39bb537071065ed1b P 841d14d044a3450dba7a52ace1d8f9f9: Bootstrap starting.
I20260807 18:33:42.616340 19754 tablet_service.cc:1467] Processing CreateTablet for tablet 35a1d137abb34ed39bb537071065ed1b (DEFAULT_TABLE table=test-table [id=2eb5c58169cd42f8b71b4c3ef26391d1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:33:42.616569 19754 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 35a1d137abb34ed39bb537071065ed1b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:33:42.617017 20000 tablet_bootstrap.cc:654] T 35a1d137abb34ed39bb537071065ed1b P 841d14d044a3450dba7a52ace1d8f9f9: Neither blocks nor log segments found. Creating new log.
I20260807 18:33:42.617105 20002 tablet_bootstrap.cc:654] T 35a1d137abb34ed39bb537071065ed1b P b8b91f4589f343c88246ca60212c015a: Neither blocks nor log segments found. Creating new log.
I20260807 18:33:42.618892 20002 tablet_bootstrap.cc:492] T 35a1d137abb34ed39bb537071065ed1b P b8b91f4589f343c88246ca60212c015a: No bootstrap required, opened a new log
I20260807 18:33:42.618978 20002 ts_tablet_manager.cc:1397] T 35a1d137abb34ed39bb537071065ed1b P b8b91f4589f343c88246ca60212c015a: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260807 18:33:42.618974 20001 tablet_bootstrap.cc:492] T 35a1d137abb34ed39bb537071065ed1b P 973fa32d73684f51949933c03a32ce8e: Bootstrap starting.
I20260807 18:33:42.619400 20002 raft_consensus.cc:348] T 35a1d137abb34ed39bb537071065ed1b P b8b91f4589f343c88246ca60212c015a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8b91f4589f343c88246ca60212c015a" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 41543 } } peers { permanent_uuid: "973fa32d73684f51949933c03a32ce8e" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 43313 } } peers { permanent_uuid: "841d14d044a3450dba7a52ace1d8f9f9" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 35051 } }
I20260807 18:33:42.619499 20002 raft_consensus.cc:374] T 35a1d137abb34ed39bb537071065ed1b P b8b91f4589f343c88246ca60212c015a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:33:42.619529 20002 raft_consensus.cc:729] T 35a1d137abb34ed39bb537071065ed1b P b8b91f4589f343c88246ca60212c015a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8b91f4589f343c88246ca60212c015a, State: Initialized, Role: FOLLOWER
I20260807 18:33:42.619684 20002 consensus_queue.cc:260] T 35a1d137abb34ed39bb537071065ed1b P b8b91f4589f343c88246ca60212c015a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8b91f4589f343c88246ca60212c015a" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 41543 } } peers { permanent_uuid: "973fa32d73684f51949933c03a32ce8e" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 43313 } } peers { permanent_uuid: "841d14d044a3450dba7a52ace1d8f9f9" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 35051 } }
I20260807 18:33:42.619864 20000 tablet_bootstrap.cc:492] T 35a1d137abb34ed39bb537071065ed1b P 841d14d044a3450dba7a52ace1d8f9f9: No bootstrap required, opened a new log
I20260807 18:33:42.619932 20001 tablet_bootstrap.cc:654] T 35a1d137abb34ed39bb537071065ed1b P 973fa32d73684f51949933c03a32ce8e: Neither blocks nor log segments found. Creating new log.
I20260807 18:33:42.619936 20002 ts_tablet_manager.cc:1428] T 35a1d137abb34ed39bb537071065ed1b P b8b91f4589f343c88246ca60212c015a: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 18:33:42.619951 20000 ts_tablet_manager.cc:1397] T 35a1d137abb34ed39bb537071065ed1b P 841d14d044a3450dba7a52ace1d8f9f9: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260807 18:33:42.620349 20000 raft_consensus.cc:348] T 35a1d137abb34ed39bb537071065ed1b P 841d14d044a3450dba7a52ace1d8f9f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8b91f4589f343c88246ca60212c015a" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 41543 } } peers { permanent_uuid: "973fa32d73684f51949933c03a32ce8e" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 43313 } } peers { permanent_uuid: "841d14d044a3450dba7a52ace1d8f9f9" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 35051 } }
I20260807 18:33:42.620476 20000 raft_consensus.cc:374] T 35a1d137abb34ed39bb537071065ed1b P 841d14d044a3450dba7a52ace1d8f9f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:33:42.620512 20000 raft_consensus.cc:729] T 35a1d137abb34ed39bb537071065ed1b P 841d14d044a3450dba7a52ace1d8f9f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 841d14d044a3450dba7a52ace1d8f9f9, State: Initialized, Role: FOLLOWER
I20260807 18:33:42.620631 20000 consensus_queue.cc:260] T 35a1d137abb34ed39bb537071065ed1b P 841d14d044a3450dba7a52ace1d8f9f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8b91f4589f343c88246ca60212c015a" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 41543 } } peers { permanent_uuid: "973fa32d73684f51949933c03a32ce8e" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 43313 } } peers { permanent_uuid: "841d14d044a3450dba7a52ace1d8f9f9" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 35051 } }
I20260807 18:33:42.620913 20000 ts_tablet_manager.cc:1428] T 35a1d137abb34ed39bb537071065ed1b P 841d14d044a3450dba7a52ace1d8f9f9: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260807 18:33:42.621519 20001 tablet_bootstrap.cc:492] T 35a1d137abb34ed39bb537071065ed1b P 973fa32d73684f51949933c03a32ce8e: No bootstrap required, opened a new log
I20260807 18:33:42.621601 20001 ts_tablet_manager.cc:1397] T 35a1d137abb34ed39bb537071065ed1b P 973fa32d73684f51949933c03a32ce8e: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260807 18:33:42.621974 20001 raft_consensus.cc:348] T 35a1d137abb34ed39bb537071065ed1b P 973fa32d73684f51949933c03a32ce8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8b91f4589f343c88246ca60212c015a" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 41543 } } peers { permanent_uuid: "973fa32d73684f51949933c03a32ce8e" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 43313 } } peers { permanent_uuid: "841d14d044a3450dba7a52ace1d8f9f9" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 35051 } }
I20260807 18:33:42.622087 20001 raft_consensus.cc:374] T 35a1d137abb34ed39bb537071065ed1b P 973fa32d73684f51949933c03a32ce8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:33:42.622126 20001 raft_consensus.cc:729] T 35a1d137abb34ed39bb537071065ed1b P 973fa32d73684f51949933c03a32ce8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 973fa32d73684f51949933c03a32ce8e, State: Initialized, Role: FOLLOWER
I20260807 18:33:42.622248 20001 consensus_queue.cc:260] T 35a1d137abb34ed39bb537071065ed1b P 973fa32d73684f51949933c03a32ce8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8b91f4589f343c88246ca60212c015a" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 41543 } } peers { permanent_uuid: "973fa32d73684f51949933c03a32ce8e" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 43313 } } peers { permanent_uuid: "841d14d044a3450dba7a52ace1d8f9f9" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 35051 } }
I20260807 18:33:42.622498 20001 ts_tablet_manager.cc:1428] T 35a1d137abb34ed39bb537071065ed1b P 973fa32d73684f51949933c03a32ce8e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 18:33:42.689802 20051 raft_consensus.cc:482] T 35a1d137abb34ed39bb537071065ed1b P 973fa32d73684f51949933c03a32ce8e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:33:42.689968 20051 raft_consensus.cc:504] T 35a1d137abb34ed39bb537071065ed1b P 973fa32d73684f51949933c03a32ce8e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8b91f4589f343c88246ca60212c015a" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 41543 } } peers { permanent_uuid: "973fa32d73684f51949933c03a32ce8e" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 43313 } } peers { permanent_uuid: "841d14d044a3450dba7a52ace1d8f9f9" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 35051 } }
I20260807 18:33:42.690433 20051 leader_election.cc:288] T 35a1d137abb34ed39bb537071065ed1b P 973fa32d73684f51949933c03a32ce8e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8b91f4589f343c88246ca60212c015a (127.11.14.65:41543), 841d14d044a3450dba7a52ace1d8f9f9 (127.11.14.67:35051)
I20260807 18:33:42.690918 19630 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "35a1d137abb34ed39bb537071065ed1b" candidate_uuid: "973fa32d73684f51949933c03a32ce8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8b91f4589f343c88246ca60212c015a" is_pre_election: true
I20260807 18:33:42.690986 19922 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "35a1d137abb34ed39bb537071065ed1b" candidate_uuid: "973fa32d73684f51949933c03a32ce8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "841d14d044a3450dba7a52ace1d8f9f9" is_pre_election: true
I20260807 18:33:42.691100 19630 raft_consensus.cc:2454] T 35a1d137abb34ed39bb537071065ed1b P b8b91f4589f343c88246ca60212c015a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 973fa32d73684f51949933c03a32ce8e in term 0.
I20260807 18:33:42.691143 19922 raft_consensus.cc:2454] T 35a1d137abb34ed39bb537071065ed1b P 841d14d044a3450dba7a52ace1d8f9f9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 973fa32d73684f51949933c03a32ce8e in term 0.
I20260807 18:33:42.691444 19706 leader_election.cc:302] T 35a1d137abb34ed39bb537071065ed1b P 973fa32d73684f51949933c03a32ce8e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 973fa32d73684f51949933c03a32ce8e, b8b91f4589f343c88246ca60212c015a; no voters:
I20260807 18:33:42.691706 20051 raft_consensus.cc:2781] T 35a1d137abb34ed39bb537071065ed1b P 973fa32d73684f51949933c03a32ce8e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:33:42.691787 20051 raft_consensus.cc:482] T 35a1d137abb34ed39bb537071065ed1b P 973fa32d73684f51949933c03a32ce8e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:33:42.691828 20051 raft_consensus.cc:3037] T 35a1d137abb34ed39bb537071065ed1b P 973fa32d73684f51949933c03a32ce8e [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:33:42.693104 20051 raft_consensus.cc:504] T 35a1d137abb34ed39bb537071065ed1b P 973fa32d73684f51949933c03a32ce8e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8b91f4589f343c88246ca60212c015a" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 41543 } } peers { permanent_uuid: "973fa32d73684f51949933c03a32ce8e" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 43313 } } peers { permanent_uuid: "841d14d044a3450dba7a52ace1d8f9f9" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 35051 } }
I20260807 18:33:42.693451 20051 leader_election.cc:288] T 35a1d137abb34ed39bb537071065ed1b P 973fa32d73684f51949933c03a32ce8e [CANDIDATE]: Term 1 election: Requested vote from peers b8b91f4589f343c88246ca60212c015a (127.11.14.65:41543), 841d14d044a3450dba7a52ace1d8f9f9 (127.11.14.67:35051)
I20260807 18:33:42.693818 19922 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "35a1d137abb34ed39bb537071065ed1b" candidate_uuid: "973fa32d73684f51949933c03a32ce8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "841d14d044a3450dba7a52ace1d8f9f9"
I20260807 18:33:42.693846 19630 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "35a1d137abb34ed39bb537071065ed1b" candidate_uuid: "973fa32d73684f51949933c03a32ce8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8b91f4589f343c88246ca60212c015a"
I20260807 18:33:42.693955 19630 raft_consensus.cc:3037] T 35a1d137abb34ed39bb537071065ed1b P b8b91f4589f343c88246ca60212c015a [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:33:42.693979 19922 raft_consensus.cc:3037] T 35a1d137abb34ed39bb537071065ed1b P 841d14d044a3450dba7a52ace1d8f9f9 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:33:42.694932 19922 raft_consensus.cc:2454] T 35a1d137abb34ed39bb537071065ed1b P 841d14d044a3450dba7a52ace1d8f9f9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 973fa32d73684f51949933c03a32ce8e in term 1.
I20260807 18:33:42.695091 19630 raft_consensus.cc:2454] T 35a1d137abb34ed39bb537071065ed1b P b8b91f4589f343c88246ca60212c015a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 973fa32d73684f51949933c03a32ce8e in term 1.
I20260807 18:33:42.695393 19706 leader_election.cc:302] T 35a1d137abb34ed39bb537071065ed1b P 973fa32d73684f51949933c03a32ce8e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 973fa32d73684f51949933c03a32ce8e, b8b91f4589f343c88246ca60212c015a; no voters:
I20260807 18:33:42.695585 20051 raft_consensus.cc:2781] T 35a1d137abb34ed39bb537071065ed1b P 973fa32d73684f51949933c03a32ce8e [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:33:42.695688 20051 raft_consensus.cc:686] T 35a1d137abb34ed39bb537071065ed1b P 973fa32d73684f51949933c03a32ce8e [term 1 LEADER]: Becoming Leader. State: Replica: 973fa32d73684f51949933c03a32ce8e, State: Running, Role: LEADER
I20260807 18:33:42.695847 20051 consensus_queue.cc:237] T 35a1d137abb34ed39bb537071065ed1b P 973fa32d73684f51949933c03a32ce8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8b91f4589f343c88246ca60212c015a" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 41543 } } peers { permanent_uuid: "973fa32d73684f51949933c03a32ce8e" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 43313 } } peers { permanent_uuid: "841d14d044a3450dba7a52ace1d8f9f9" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 35051 } }
I20260807 18:33:42.697572 19465 catalog_manager.cc:5495] T 35a1d137abb34ed39bb537071065ed1b P 973fa32d73684f51949933c03a32ce8e reported cstate change: term changed from 0 to 1, leader changed from <none> to 973fa32d73684f51949933c03a32ce8e (127.11.14.66). New cstate: current_term: 1 leader_uuid: "973fa32d73684f51949933c03a32ce8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8b91f4589f343c88246ca60212c015a" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 41543 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "973fa32d73684f51949933c03a32ce8e" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 43313 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "841d14d044a3450dba7a52ace1d8f9f9" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 35051 } health_report { overall_health: UNKNOWN } } }
I20260807 18:33:42.734825 19997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:42.729289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41834 (local address 127.11.14.66:43313)
0807 18:33:42.729434 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:42.729440 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:42.729976 (+ 536us) server_negotiation.cc:407] Connection header received
0807 18:33:42.730053 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:42.730056 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:42.730124 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:42.730210 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:33:42.730680 (+ 470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:42.731549 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:42.732955 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:42.733325 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:42.733475 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:42.733995 (+ 520us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:33:42.734093 (+ 98us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:33:42.734179 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:42.734565 (+ 386us) server_negotiation.cc:299] Negotiation successful
0807 18:33:42.734663 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260807 18:33:42.746366 19997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:42.742232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.66:49385 (local address 127.11.14.66:43313)
0807 18:33:42.742348 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:42.742354 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:42.742370 (+ 16us) server_negotiation.cc:407] Connection header received
0807 18:33:42.742418 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:42.742422 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:42.742477 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:42.742566 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:33:42.742949 (+ 383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:42.743767 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:42.745002 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:42.745347 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:42.745447 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:42.745815 (+ 368us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:33:42.745902 (+ 87us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:33:42.745984 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:42.746152 (+ 168us) server_negotiation.cc:299] Negotiation successful
0807 18:33:42.746203 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260807 18:33:42.746410 20059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:42.741698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.66:43313 (local address 127.11.14.66:49385)
0807 18:33:42.741894 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:42.741912 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:42.742005 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:42.742703 (+ 698us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:42.742707 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:42.742716 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 18:33:42.742807 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:42.742815 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:42.743914 (+ 1099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:42.743918 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:42.744860 (+ 942us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:42.744875 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:42.745505 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:42.745510 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:42.745626 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:42.745669 (+ 43us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 18:33:42.746030 (+ 361us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 18:33:42.746040 (+ 10us) client_negotiation.cc:769] Sending connection context
0807 18:33:42.746134 (+ 94us) client_negotiation.cc:240] Negotiation successful
0807 18:33:42.746225 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105}
I20260807 18:33:42.753453 19630 raft_consensus.cc:1261] T 35a1d137abb34ed39bb537071065ed1b P b8b91f4589f343c88246ca60212c015a [term 1 FOLLOWER]: Refusing update from remote peer 973fa32d73684f51949933c03a32ce8e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:33:42.753605 19922 raft_consensus.cc:1261] T 35a1d137abb34ed39bb537071065ed1b P 841d14d044a3450dba7a52ace1d8f9f9 [term 1 FOLLOWER]: Refusing update from remote peer 973fa32d73684f51949933c03a32ce8e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:33:42.753928 20051 consensus_queue.cc:1035] T 35a1d137abb34ed39bb537071065ed1b P 973fa32d73684f51949933c03a32ce8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8b91f4589f343c88246ca60212c015a" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 41543 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:33:42.754127 20043 consensus_queue.cc:1035] T 35a1d137abb34ed39bb537071065ed1b P 973fa32d73684f51949933c03a32ce8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "841d14d044a3450dba7a52ace1d8f9f9" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 35051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:33:42.795768 19465 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52826:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:33:42.795956 19465 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52826:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:33:42.798099 19465 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8482b9c6ca944b908d05b969396896b8: Sending DeleteTablet for 3 replicas of tablet 35a1d137abb34ed39bb537071065ed1b
I20260807 18:33:42.798688 19753 tablet_service.cc:1514] Processing DeleteTablet for tablet 35a1d137abb34ed39bb537071065ed1b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:33:42 UTC) from {username='oryx', principal='oryx/127.11.14.126@KRBTEST.COM'} at 127.0.0.1:41810
I20260807 18:33:42.798740 19902 tablet_service.cc:1514] Processing DeleteTablet for tablet 35a1d137abb34ed39bb537071065ed1b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:33:42 UTC) from {username='oryx', principal='oryx/127.11.14.126@KRBTEST.COM'} at 127.0.0.1:58600
I20260807 18:33:42.798918 20024 tablet_replica.cc:330] T 35a1d137abb34ed39bb537071065ed1b P 841d14d044a3450dba7a52ace1d8f9f9: stopping tablet replica
I20260807 18:33:42.798897 19610 tablet_service.cc:1514] Processing DeleteTablet for tablet 35a1d137abb34ed39bb537071065ed1b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:33:42 UTC) from {username='oryx', principal='oryx/127.11.14.126@KRBTEST.COM'} at 127.0.0.1:45022
I20260807 18:33:42.799044 20024 raft_consensus.cc:2229] T 35a1d137abb34ed39bb537071065ed1b P 841d14d044a3450dba7a52ace1d8f9f9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:33:42.799070 20022 tablet_replica.cc:330] T 35a1d137abb34ed39bb537071065ed1b P 973fa32d73684f51949933c03a32ce8e: stopping tablet replica
I20260807 18:33:42.799173 20024 raft_consensus.cc:2258] T 35a1d137abb34ed39bb537071065ed1b P 841d14d044a3450dba7a52ace1d8f9f9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:33:42.799166 20022 raft_consensus.cc:2229] T 35a1d137abb34ed39bb537071065ed1b P 973fa32d73684f51949933c03a32ce8e [term 1 LEADER]: Raft consensus shutting down.
I20260807 18:33:42.799446 20022 raft_consensus.cc:2258] T 35a1d137abb34ed39bb537071065ed1b P 973fa32d73684f51949933c03a32ce8e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:33:42.799669 20025 tablet_replica.cc:330] T 35a1d137abb34ed39bb537071065ed1b P b8b91f4589f343c88246ca60212c015a: stopping tablet replica
I20260807 18:33:42.799785 20025 raft_consensus.cc:2229] T 35a1d137abb34ed39bb537071065ed1b P b8b91f4589f343c88246ca60212c015a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:33:42.799916 20025 raft_consensus.cc:2258] T 35a1d137abb34ed39bb537071065ed1b P b8b91f4589f343c88246ca60212c015a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:33:42.800685 20025 ts_tablet_manager.cc:1905] T 35a1d137abb34ed39bb537071065ed1b P b8b91f4589f343c88246ca60212c015a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:33:42.800806 20022 ts_tablet_manager.cc:1905] T 35a1d137abb34ed39bb537071065ed1b P 973fa32d73684f51949933c03a32ce8e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:33:42.801198 11321 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkz56XN/build/debug/bin/kudu with pid 19541
I20260807 18:33:42.802346 20024 ts_tablet_manager.cc:1905] T 35a1d137abb34ed39bb537071065ed1b P 841d14d044a3450dba7a52ace1d8f9f9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:33:42.804687 20022 ts_tablet_manager.cc:1918] T 35a1d137abb34ed39bb537071065ed1b P 973fa32d73684f51949933c03a32ce8e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 18:33:42.804841 20022 log.cc:1196] T 35a1d137abb34ed39bb537071065ed1b P 973fa32d73684f51949933c03a32ce8e: Deleting WAL directory at /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127579900523-11321-0/minicluster-data/ts-1/wal/wals/35a1d137abb34ed39bb537071065ed1b
I20260807 18:33:42.805218 20022 ts_tablet_manager.cc:1939] T 35a1d137abb34ed39bb537071065ed1b P 973fa32d73684f51949933c03a32ce8e: Deleting consensus metadata
I20260807 18:33:42.805744 20024 ts_tablet_manager.cc:1918] T 35a1d137abb34ed39bb537071065ed1b P 841d14d044a3450dba7a52ace1d8f9f9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 18:33:42.805867 20024 log.cc:1196] T 35a1d137abb34ed39bb537071065ed1b P 841d14d044a3450dba7a52ace1d8f9f9: Deleting WAL directory at /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786127579900523-11321-0/minicluster-data/ts-2/wal/wals/35a1d137abb34ed39bb537071065ed1b
I20260807 18:33:42.806167 20024 ts_tablet_manager.cc:1939] T 35a1d137abb34ed39bb537071065ed1b P 841d14d044a3450dba7a52ace1d8f9f9: Deleting consensus metadata
I20260807 18:33:42.806149 19450 catalog_manager.cc:4841] TS 973fa32d73684f51949933c03a32ce8e (127.11.14.66:43313): tablet 35a1d137abb34ed39bb537071065ed1b (table test-table [id=2eb5c58169cd42f8b71b4c3ef26391d1]) successfully deleted
I20260807 18:33:42.806931 19451 catalog_manager.cc:4841] TS 841d14d044a3450dba7a52ace1d8f9f9 (127.11.14.67:35051): tablet 35a1d137abb34ed39bb537071065ed1b (table test-table [id=2eb5c58169cd42f8b71b4c3ef26391d1]) successfully deleted
W20260807 18:33:42.815303 19451 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.14.65:41543 (error 108)
I20260807 18:33:42.815616 11321 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkz56XN/build/debug/bin/kudu with pid 19687
I20260807 18:33:42.818397 20098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:42.818008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.65:41543 (local address 127.0.0.1:45032)
0807 18:33:42.818220 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:42.818287 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.65:41543: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":65,"threads_started":1}
W20260807 18:33:42.818641 19451 catalog_manager.cc:4584] TS b8b91f4589f343c88246ca60212c015a (127.11.14.65:41543): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 35a1d137abb34ed39bb537071065ed1b: Network error: Client connection negotiation failed: client connection to 127.11.14.65:41543: connect: Connection refused (error 111)
I20260807 18:33:42.825873 11321 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkz56XN/build/debug/bin/kudu with pid 19833
I20260807 18:33:42.835093 11321 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkz56XN/build/debug/bin/kudu with pid 19433
2026-08-07T18:33:42Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (3417 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786127579900523-11321-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:33:42 dist-test-slave-6k22 krb5kdc[20101](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:33:42 dist-test-slave-6k22 krb5kdc[20101](info): set up 2 sockets
Aug 07 18:33:42 dist-test-slave-6k22 krb5kdc[20101](info): commencing operation
krb5kdc: starting...
W20260807 18:33:43.947319 11321 mini_kdc.cc:121] Time spent starting KDC: real 1.086s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786127579900523-11321-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786127579900523-11321-0/krb5kdc/test-user.keytab.
Aug 07 18:33:44 dist-test-slave-6k22 krb5kdc[20101](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127624, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:33:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:33:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.14.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.14.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.14.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.126.keytab.
Entry for principal kudu/127.11.14.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.126.keytab.
I20260807 18:33:44.076530 11321 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkz56XN/build/debug/bin/kudu
/tmp/dist-test-taskkz56XN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786127579900523-11321-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786127579900523-11321-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786127579900523-11321-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786127579900523-11321-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.14.126:35865
--webserver_interface=127.11.14.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.126.keytab
--principal=kudu/127.11.14.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.14.84:43875
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.14.126:35865
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786127579900523-11321-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786127579900523-11321-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786127579900523-11321-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:33:44.193835 20115 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:33:44.194077 20115 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:33:44.194115 20115 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:33:44.198107 20115 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:33:44.198167 20115 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:33:44.198187 20115 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:33:44.198204 20115 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:33:44.198220 20115 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:33:44.203375 20115 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.14.84:43875
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786127579900523-11321-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786127579900523-11321-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.14.126:35865
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.126.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.14.126:35865
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786127579900523-11321-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.14.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20115
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786127579900523-11321-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:33:44.204407 20115 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:33:44.205365 20115 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:33:44.213907 20122 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:33:44.213907 20121 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:33:44.213958 20124 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:33:44.214114 20115 server_base.cc:1034] running on GCE node
I20260807 18:33:44.214838 20115 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:33:44.216034 20115 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:33:44.217231 20115 hybrid_clock.cc:648] HybridClock initialized: now 1786127624217203 us; error 43 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-07T18:33:44Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1395 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127579900523-11321-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:33:44 dist-test-slave-6k22 krb5kdc[20129](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:33:44 dist-test-slave-6k22 krb5kdc[20129](info): set up 2 sockets
Aug 07 18:33:44 dist-test-slave-6k22 krb5kdc[20129](info): commencing operation
krb5kdc: starting...
W20260807 18:33:45.375263 11321 mini_kdc.cc:121] Time spent starting KDC: real 1.118s user 0.005s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127579900523-11321-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127579900523-11321-0/krb5kdc/test-user.keytab.
Aug 07 18:33:45 dist-test-slave-6k22 krb5kdc[20129](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127625, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:33:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:33:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.14.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.14.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.14.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.126.keytab.
Entry for principal kudu/127.11.14.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.126.keytab.
I20260807 18:33:45.500720 11321 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkz56XN/build/debug/bin/kudu
/tmp/dist-test-taskkz56XN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127579900523-11321-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127579900523-11321-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127579900523-11321-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127579900523-11321-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.14.126:43679
--webserver_interface=127.11.14.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.126.keytab
--principal=kudu/127.11.14.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.14.84:45823
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.14.126:43679
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127579900523-11321-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127579900523-11321-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127579900523-11321-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:33:45.614200 20143 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:33:45.614497 20143 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:33:45.614573 20143 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:33:45.619073 20143 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:33:45.619163 20143 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:33:45.619201 20143 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:33:45.619239 20143 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:33:45.619277 20143 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:33:45.624529 20143 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.14.84:45823
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127579900523-11321-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127579900523-11321-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.14.126:43679
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.126.keytab
--principal=kudu/127.11.14.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.14.126:43679
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127579900523-11321-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.14.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20143
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127579900523-11321-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:33:45.625594 20143 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:33:45.626626 20143 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:33:45.635394 20149 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:33:45.635344 20152 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:33:45.635344 20150 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:33:45.635658 20143 server_base.cc:1034] running on GCE node
I20260807 18:33:45.636238 20143 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:33:45.637364 20143 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:33:45.638557 20143 hybrid_clock.cc:648] HybridClock initialized: now 1786127625638525 us; error 43 us; skew 500 ppm
Aug 07 18:33:45 dist-test-slave-6k22 krb5kdc[20129](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127625, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:33:45.641467 20143 init.cc:377] Logged in from keytab as kudu/127.11.14.126@KRBTEST.COM (short username kudu)
I20260807 18:33:45.642307 20143 webserver.cc:460] Webserver started at http://127.11.14.126:45961/ using document root <none> and password file <none>
I20260807 18:33:45.642997 20143 fs_manager.cc:359] Metadata directory not provided
I20260807 18:33:45.643065 20143 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:33:45.643302 20143 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:33:45.645453 20143 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127579900523-11321-0/minicluster-data/master-0/data/instance:
uuid: "15d605430db248bdbce86be96eb7d04a"
format_stamp: "Formatted at 2026-08-07 18:33:45 on dist-test-slave-6k22"
server_key: "60b90d58c888dcc52b0d48f9f3de33d7"
server_key_iv: "6299dc0e7cfbbadc6c1af995deca7220"
server_key_version: "encryptionkey@0"
I20260807 18:33:45.646044 20143 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127579900523-11321-0/minicluster-data/master-0/wal/instance:
uuid: "15d605430db248bdbce86be96eb7d04a"
format_stamp: "Formatted at 2026-08-07 18:33:45 on dist-test-slave-6k22"
server_key: "60b90d58c888dcc52b0d48f9f3de33d7"
server_key_iv: "6299dc0e7cfbbadc6c1af995deca7220"
server_key_version: "encryptionkey@0"
I20260807 18:33:45.649926 20143 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260807 18:33:45.652388 20159 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:33:45.653617 20143 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260807 18:33:45.653765 20143 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127579900523-11321-0/minicluster-data/master-0/data,/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127579900523-11321-0/minicluster-data/master-0/wal
uuid: "15d605430db248bdbce86be96eb7d04a"
format_stamp: "Formatted at 2026-08-07 18:33:45 on dist-test-slave-6k22"
server_key: "60b90d58c888dcc52b0d48f9f3de33d7"
server_key_iv: "6299dc0e7cfbbadc6c1af995deca7220"
server_key_version: "encryptionkey@0"
I20260807 18:33:45.653885 20143 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127579900523-11321-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127579900523-11321-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127579900523-11321-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:33:45.664824 20143 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:33:45.667695 20143 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:33:45.667886 20143 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:33:45.675099 20143 rpc_server.cc:307] RPC server started. Bound to: 127.11.14.126:43679
I20260807 18:33:45.675168 20214 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.14.126:43679 every 8 connection(s)
I20260807 18:33:45.676179 20143 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127579900523-11321-0/minicluster-data/master-0/data/info.pb
I20260807 18:33:45.677034 11321 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkz56XN/build/debug/bin/kudu as pid 20143
I20260807 18:33:45.677188 11321 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127579900523-11321-0/minicluster-data/master-0/wal/instance
I20260807 18:33:45.677512 11321 external_mini_cluster.cc:1428] Setting key 4a932772e2a2f6ef012762d3d9f419fd
I20260807 18:33:45.679533 20215 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:33:45.686546 20215 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 15d605430db248bdbce86be96eb7d04a: Bootstrap starting.
Aug 07 18:33:45 dist-test-slave-6k22 krb5kdc[20129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127625, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.14.126@KRBTEST.COM
I20260807 18:33:45.689462 20215 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 15d605430db248bdbce86be96eb7d04a: Neither blocks nor log segments found. Creating new log.
I20260807 18:33:45.690399 20215 log.cc:824] T 00000000000000000000000000000000 P 15d605430db248bdbce86be96eb7d04a: Log is configured to *not* fsync() on all Append() calls
I20260807 18:33:45.692699 20215 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 15d605430db248bdbce86be96eb7d04a: No bootstrap required, opened a new log
I20260807 18:33:45.692987 20218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:45.679022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40456 (local address 127.11.14.126:43679)
0807 18:33:45.679464 (+ 442us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:45.679483 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:45.679527 (+ 44us) server_negotiation.cc:407] Connection header received
0807 18:33:45.680425 (+ 898us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:45.680448 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:45.680877 (+ 429us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:45.681286 (+ 409us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:33:45.681926 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:45.683440 (+ 1514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:45.684351 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:45.685046 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:45.685205 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:45.687791 (+ 2586us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:33:45.687798 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:33:45.687820 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:33:45.687858 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:33:45.689853 (+ 1995us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:45.690274 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:45.690282 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:45.690289 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:45.690443 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:45.690741 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:45.690747 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:45.690750 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:45.691219 (+ 469us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:33:45.691422 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:45.691685 (+ 263us) server_negotiation.cc:299] Negotiation successful
0807 18:33:45.691977 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":283,"thread_start_us":124,"threads_started":1}
I20260807 18:33:45.697022 20215 raft_consensus.cc:348] T 00000000000000000000000000000000 P 15d605430db248bdbce86be96eb7d04a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15d605430db248bdbce86be96eb7d04a" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 43679 } }
I20260807 18:33:45.697273 20215 raft_consensus.cc:374] T 00000000000000000000000000000000 P 15d605430db248bdbce86be96eb7d04a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:33:45.697333 20215 raft_consensus.cc:729] T 00000000000000000000000000000000 P 15d605430db248bdbce86be96eb7d04a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 15d605430db248bdbce86be96eb7d04a, State: Initialized, Role: FOLLOWER
I20260807 18:33:45.697788 20215 consensus_queue.cc:260] T 00000000000000000000000000000000 P 15d605430db248bdbce86be96eb7d04a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15d605430db248bdbce86be96eb7d04a" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 43679 } }
I20260807 18:33:45.697954 20215 raft_consensus.cc:388] T 00000000000000000000000000000000 P 15d605430db248bdbce86be96eb7d04a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:33:45.698021 20215 raft_consensus.cc:482] T 00000000000000000000000000000000 P 15d605430db248bdbce86be96eb7d04a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:33:45.698119 20215 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 15d605430db248bdbce86be96eb7d04a [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:33:45.699046 20215 raft_consensus.cc:504] T 00000000000000000000000000000000 P 15d605430db248bdbce86be96eb7d04a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15d605430db248bdbce86be96eb7d04a" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 43679 } }
I20260807 18:33:45.699396 20215 leader_election.cc:302] T 00000000000000000000000000000000 P 15d605430db248bdbce86be96eb7d04a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 15d605430db248bdbce86be96eb7d04a; no voters:
I20260807 18:33:45.699743 20215 leader_election.cc:288] T 00000000000000000000000000000000 P 15d605430db248bdbce86be96eb7d04a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:33:45.699890 20220 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 15d605430db248bdbce86be96eb7d04a [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:33:45.700140 20220 raft_consensus.cc:686] T 00000000000000000000000000000000 P 15d605430db248bdbce86be96eb7d04a [term 1 LEADER]: Becoming Leader. State: Replica: 15d605430db248bdbce86be96eb7d04a, State: Running, Role: LEADER
I20260807 18:33:45.700552 20220 consensus_queue.cc:237] T 00000000000000000000000000000000 P 15d605430db248bdbce86be96eb7d04a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15d605430db248bdbce86be96eb7d04a" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 43679 } }
I20260807 18:33:45.700989 20215 sys_catalog.cc:564] T 00000000000000000000000000000000 P 15d605430db248bdbce86be96eb7d04a [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:33:45.702299 20221 sys_catalog.cc:455] T 00000000000000000000000000000000 P 15d605430db248bdbce86be96eb7d04a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "15d605430db248bdbce86be96eb7d04a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15d605430db248bdbce86be96eb7d04a" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 43679 } } }
I20260807 18:33:45.702320 20223 sys_catalog.cc:455] T 00000000000000000000000000000000 P 15d605430db248bdbce86be96eb7d04a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 15d605430db248bdbce86be96eb7d04a. Latest consensus state: current_term: 1 leader_uuid: "15d605430db248bdbce86be96eb7d04a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15d605430db248bdbce86be96eb7d04a" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 43679 } } }
I20260807 18:33:45.702451 20221 sys_catalog.cc:458] T 00000000000000000000000000000000 P 15d605430db248bdbce86be96eb7d04a [sys.catalog]: This master's current role is: LEADER
I20260807 18:33:45.702455 20223 sys_catalog.cc:458] T 00000000000000000000000000000000 P 15d605430db248bdbce86be96eb7d04a [sys.catalog]: This master's current role is: LEADER
I20260807 18:33:45.702853 20230 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:33:45.706118 20230 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:33:45.711738 20230 catalog_manager.cc:1347] Generated new cluster ID: 19583b40706a4986b2070159ee990882
I20260807 18:33:45.711800 20230 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:33:45.735832 20230 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:33:45.737246 20230 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:33:45.749584 20230 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 15d605430db248bdbce86be96eb7d04a: Generated new TSK 0
I20260807 18:33:45.750537 20230 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.14.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.14.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.14.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.65.keytab.
Entry for principal kudu/127.11.14.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.65.keytab.
I20260807 18:33:45.786010 11321 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkz56XN/build/debug/bin/kudu
/tmp/dist-test-taskkz56XN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127579900523-11321-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127579900523-11321-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127579900523-11321-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127579900523-11321-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.14.65:0
--local_ip_for_outbound_sockets=127.11.14.65
--webserver_interface=127.11.14.65
--webserver_port=0
--tserver_master_addrs=127.11.14.126:43679
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.65.keytab
--principal=kudu/127.11.14.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.14.84:45823
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127579900523-11321-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127579900523-11321-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127579900523-11321-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:33:45.900223 20243 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:33:45.900507 20243 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:33:45.900580 20243 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:33:45.904861 20243 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:33:45.904943 20243 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:33:45.905032 20243 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.14.65
I20260807 18:33:45.910920 20243 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.14.84:45823
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127579900523-11321-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127579900523-11321-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.65.keytab
--principal=kudu/127.11.14.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.14.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127579900523-11321-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.14.65
--webserver_port=0
--tserver_master_addrs=127.11.14.126:43679
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20243
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.14.65
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127579900523-11321-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:33:45.912132 20243 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:33:45.913275 20243 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:33:45.923679 20250 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:33:45.923686 20252 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:33:45.923687 20249 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:33:45.923972 20243 server_base.cc:1034] running on GCE node
I20260807 18:33:45.924441 20243 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:33:45.925179 20243 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:33:45.926393 20243 hybrid_clock.cc:648] HybridClock initialized: now 1786127625926371 us; error 33 us; skew 500 ppm
Aug 07 18:33:45 dist-test-slave-6k22 krb5kdc[20129](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127625, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:33:45.929215 20243 init.cc:377] Logged in from keytab as kudu/127.11.14.65@KRBTEST.COM (short username kudu)
I20260807 18:33:45.930161 20243 webserver.cc:460] Webserver started at http://127.11.14.65:38617/ using document root <none> and password file <none>
I20260807 18:33:45.930737 20243 fs_manager.cc:359] Metadata directory not provided
I20260807 18:33:45.930835 20243 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:33:45.931082 20243 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:33:45.933202 20243 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127579900523-11321-0/minicluster-data/ts-0/data/instance:
uuid: "efb72bd369324b0fa9a81ea56bb2ca2e"
format_stamp: "Formatted at 2026-08-07 18:33:45 on dist-test-slave-6k22"
server_key: "7232ee7d96b1457e65b652a9ab6552d1"
server_key_iv: "741b66dec5bcd1eea1b0d0d6ca2ac688"
server_key_version: "encryptionkey@0"
I20260807 18:33:45.933784 20243 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127579900523-11321-0/minicluster-data/ts-0/wal/instance:
uuid: "efb72bd369324b0fa9a81ea56bb2ca2e"
format_stamp: "Formatted at 2026-08-07 18:33:45 on dist-test-slave-6k22"
server_key: "7232ee7d96b1457e65b652a9ab6552d1"
server_key_iv: "741b66dec5bcd1eea1b0d0d6ca2ac688"
server_key_version: "encryptionkey@0"
I20260807 18:33:45.937645 20243 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 18:33:45.940191 20259 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:33:45.941581 20243 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:33:45.941798 20243 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127579900523-11321-0/minicluster-data/ts-0/data,/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127579900523-11321-0/minicluster-data/ts-0/wal
uuid: "efb72bd369324b0fa9a81ea56bb2ca2e"
format_stamp: "Formatted at 2026-08-07 18:33:45 on dist-test-slave-6k22"
server_key: "7232ee7d96b1457e65b652a9ab6552d1"
server_key_iv: "741b66dec5bcd1eea1b0d0d6ca2ac688"
server_key_version: "encryptionkey@0"
I20260807 18:33:45.941932 20243 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127579900523-11321-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127579900523-11321-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127579900523-11321-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:33:45.979760 20243 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:33:45.982843 20243 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:33:45.983074 20243 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:33:45.983733 20243 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:33:45.984601 20243 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:33:45.984691 20243 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:33:45.984766 20243 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:33:45.984814 20243 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:33:45.994154 20243 rpc_server.cc:307] RPC server started. Bound to: 127.11.14.65:42313
I20260807 18:33:45.994199 20379 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.14.65:42313 every 8 connection(s)
I20260807 18:33:45.995069 20243 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127579900523-11321-0/minicluster-data/ts-0/data/info.pb
I20260807 18:33:46.003132 11321 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkz56XN/build/debug/bin/kudu as pid 20243
I20260807 18:33:46.003289 11321 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127579900523-11321-0/minicluster-data/ts-0/wal/instance
I20260807 18:33:46.003670 11321 external_mini_cluster.cc:1428] Setting key 5818c457bc9b6f544f9c7883814f78fb
Aug 07 18:33:46 dist-test-slave-6k22 krb5kdc[20129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127625, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.65@KRBTEST.COM for kudu/127.11.14.126@KRBTEST.COM
I20260807 18:33:46.009730 20218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:45.996997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.65:41781 (local address 127.11.14.126:43679)
0807 18:33:45.997246 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:45.997251 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:45.998026 (+ 775us) server_negotiation.cc:407] Connection header received
0807 18:33:45.999385 (+ 1359us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:45.999389 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:45.999448 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:45.999558 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:33:46.000845 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:46.001421 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:46.002712 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:46.002984 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:46.003116 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:46.006043 (+ 2927us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:33:46.006052 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:33:46.006054 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:33:46.006077 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:33:46.007518 (+ 1441us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:46.008080 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:46.008087 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:46.008089 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:46.008175 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:46.008541 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:46.008547 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:46.008549 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:46.008739 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:33:46.008864 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:46.009495 (+ 631us) server_negotiation.cc:299] Negotiation successful
0807 18:33:46.009606 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260807 18:33:46.011029 20383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:45.997390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:43679 (local address 127.11.14.65:41781)
0807 18:33:45.997863 (+ 473us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:45.997909 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:45.999147 (+ 1238us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:45.999769 (+ 622us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:45.999775 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:46.000136 (+ 361us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:33:46.000694 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:46.000710 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:46.001554 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:46.001564 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:46.002572 (+ 1008us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:46.002582 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:46.003155 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:46.003160 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:46.003416 (+ 256us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:46.004139 (+ 723us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:33:46.004159 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:33:46.005869 (+ 1710us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:33:46.007670 (+ 1801us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:33:46.007676 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:33:46.007693 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:33:46.007967 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:33:46.008276 (+ 309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:33:46.008279 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:33:46.008281 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:33:46.008437 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:33:46.008854 (+ 417us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:33:46.008860 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:33:46.009197 (+ 337us) client_negotiation.cc:769] Sending connection context
0807 18:33:46.009433 (+ 236us) client_negotiation.cc:240] Negotiation successful
0807 18:33:46.009821 (+ 388us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":108,"threads_started":1}
I20260807 18:33:46.012367 20380 heartbeater.cc:347] Connected to a master server at 127.11.14.126:43679
I20260807 18:33:46.012660 20380 heartbeater.cc:464] Registering TS with master...
I20260807 18:33:46.013298 20380 heartbeater.cc:511] Master 127.11.14.126:43679 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.11.14.66@KRBTEST.COM; defaulting to no policy
I20260807 18:33:46.015117 20173 ts_manager.cc:194] Registered new tserver with Master: efb72bd369324b0fa9a81ea56bb2ca2e (127.11.14.65:42313)
I20260807 18:33:46.016568 20173 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.14.65@KRBTEST.COM'} at 127.11.14.65:41781
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.14.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.14.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.66.keytab.
Entry for principal kudu/127.11.14.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.66.keytab.
I20260807 18:33:46.035945 11321 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkz56XN/build/debug/bin/kudu
/tmp/dist-test-taskkz56XN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127579900523-11321-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127579900523-11321-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127579900523-11321-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127579900523-11321-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.14.66:0
--local_ip_for_outbound_sockets=127.11.14.66
--webserver_interface=127.11.14.66
--webserver_port=0
--tserver_master_addrs=127.11.14.126:43679
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.66.keytab
--principal=kudu/127.11.14.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.14.84:45823
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127579900523-11321-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127579900523-11321-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127579900523-11321-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:33:46.159310 20386 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:33:46.159669 20386 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:33:46.159734 20386 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:33:46.163823 20386 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:33:46.163892 20386 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:33:46.163954 20386 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.14.66
I20260807 18:33:46.169292 20386 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.14.84:45823
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127579900523-11321-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127579900523-11321-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.66.keytab
--principal=kudu/127.11.14.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.14.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127579900523-11321-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.14.66
--webserver_port=0
--tserver_master_addrs=127.11.14.126:43679
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20386
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.14.66
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127579900523-11321-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:33:46.170405 20386 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:33:46.171435 20386 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:33:46.184367 20392 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:33:46.184331 20394 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:33:46.184346 20391 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:33:46.184588 20386 server_base.cc:1034] running on GCE node
I20260807 18:33:46.185169 20386 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:33:46.185905 20386 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:33:46.187098 20386 hybrid_clock.cc:648] HybridClock initialized: now 1786127626187075 us; error 36 us; skew 500 ppm
Aug 07 18:33:46 dist-test-slave-6k22 krb5kdc[20129](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127626, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:33:46.189998 20386 init.cc:377] Logged in from keytab as kudu/127.11.14.66@KRBTEST.COM (short username kudu)
I20260807 18:33:46.190949 20386 webserver.cc:460] Webserver started at http://127.11.14.66:33585/ using document root <none> and password file <none>
I20260807 18:33:46.191618 20386 fs_manager.cc:359] Metadata directory not provided
I20260807 18:33:46.191712 20386 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:33:46.192005 20386 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:33:46.194161 20386 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127579900523-11321-0/minicluster-data/ts-1/data/instance:
uuid: "f84e9ea34d3a47d8952e0e84104765c5"
format_stamp: "Formatted at 2026-08-07 18:33:46 on dist-test-slave-6k22"
server_key: "b2cf9757edfbf58995b9313bf5d59a33"
server_key_iv: "a9eb0082a48e55c25f3eff1ad2244974"
server_key_version: "encryptionkey@0"
I20260807 18:33:46.194751 20386 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127579900523-11321-0/minicluster-data/ts-1/wal/instance:
uuid: "f84e9ea34d3a47d8952e0e84104765c5"
format_stamp: "Formatted at 2026-08-07 18:33:46 on dist-test-slave-6k22"
server_key: "b2cf9757edfbf58995b9313bf5d59a33"
server_key_iv: "a9eb0082a48e55c25f3eff1ad2244974"
server_key_version: "encryptionkey@0"
I20260807 18:33:46.198588 20386 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 18:33:46.201274 20401 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:33:46.202826 20386 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 18:33:46.202960 20386 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127579900523-11321-0/minicluster-data/ts-1/data,/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127579900523-11321-0/minicluster-data/ts-1/wal
uuid: "f84e9ea34d3a47d8952e0e84104765c5"
format_stamp: "Formatted at 2026-08-07 18:33:46 on dist-test-slave-6k22"
server_key: "b2cf9757edfbf58995b9313bf5d59a33"
server_key_iv: "a9eb0082a48e55c25f3eff1ad2244974"
server_key_version: "encryptionkey@0"
I20260807 18:33:46.203074 20386 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127579900523-11321-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127579900523-11321-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127579900523-11321-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:33:46.217185 20386 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:33:46.220223 20386 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:33:46.220448 20386 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:33:46.221071 20386 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:33:46.221942 20386 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:33:46.222018 20386 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:33:46.222091 20386 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:33:46.222144 20386 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:33:46.232084 20386 rpc_server.cc:307] RPC server started. Bound to: 127.11.14.66:42163
I20260807 18:33:46.232060 20520 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.14.66:42163 every 8 connection(s)
I20260807 18:33:46.233209 20386 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127579900523-11321-0/minicluster-data/ts-1/data/info.pb
Aug 07 18:33:46 dist-test-slave-6k22 krb5kdc[20129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127626, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.66@KRBTEST.COM for kudu/127.11.14.126@KRBTEST.COM
I20260807 18:33:46.243520 11321 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkz56XN/build/debug/bin/kudu as pid 20386
I20260807 18:33:46.243693 11321 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127579900523-11321-0/minicluster-data/ts-1/wal/instance
I20260807 18:33:46.244081 11321 external_mini_cluster.cc:1428] Setting key 98e5bd7dc7d1dfa3bf931b11dfffb019
I20260807 18:33:46.247846 20218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:46.235140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.66:47753 (local address 127.11.14.126:43679)
0807 18:33:46.235295 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:46.235299 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:46.236204 (+ 905us) server_negotiation.cc:407] Connection header received
0807 18:33:46.237564 (+ 1360us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:46.237567 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:46.237624 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:46.237706 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:33:46.238918 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:46.239476 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:46.240652 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:46.240884 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:46.240997 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:46.243652 (+ 2655us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:33:46.243658 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:33:46.243664 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:33:46.243713 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:33:46.245502 (+ 1789us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:46.246140 (+ 638us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:46.246144 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:46.246146 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:46.246244 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:46.246639 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:46.246643 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:46.246645 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:46.246860 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:33:46.246975 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:46.247509 (+ 534us) server_negotiation.cc:299] Negotiation successful
0807 18:33:46.247678 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260807 18:33:46.248729 20523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:46.235537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:43679 (local address 127.11.14.66:47753)
0807 18:33:46.236051 (+ 514us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:46.236096 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:46.237343 (+ 1247us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:46.237890 (+ 547us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:46.237897 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:46.238256 (+ 359us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:33:46.238761 (+ 505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:46.238774 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:46.239643 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:46.239647 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:46.240522 (+ 875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:46.240532 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:46.241043 (+ 511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:46.241048 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:46.241214 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:46.241874 (+ 660us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:33:46.241890 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:33:46.243398 (+ 1508us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:33:46.245627 (+ 2229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:33:46.245633 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:33:46.245650 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:33:46.246015 (+ 365us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:33:46.246343 (+ 328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:33:46.246346 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:33:46.246347 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:33:46.246507 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:33:46.246975 (+ 468us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:33:46.246984 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:33:46.247251 (+ 267us) client_negotiation.cc:769] Sending connection context
0807 18:33:46.247467 (+ 216us) client_negotiation.cc:240] Negotiation successful
0807 18:33:46.247851 (+ 384us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":268,"thread_start_us":118,"threads_started":1}
I20260807 18:33:46.250051 20521 heartbeater.cc:347] Connected to a master server at 127.11.14.126:43679
I20260807 18:33:46.250344 20521 heartbeater.cc:464] Registering TS with master...
I20260807 18:33:46.250897 20521 heartbeater.cc:511] Master 127.11.14.126:43679 requested a full tablet report, sending...
I20260807 18:33:46.252116 20173 ts_manager.cc:194] Registered new tserver with Master: f84e9ea34d3a47d8952e0e84104765c5 (127.11.14.66:42163)
I20260807 18:33:46.252698 20173 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.14.66@KRBTEST.COM'} at 127.11.14.66:47753
WARNING: no policy specified for kudu/127.11.14.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.14.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.14.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.67.keytab.
Entry for principal kudu/127.11.14.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.67.keytab.
I20260807 18:33:46.275401 11321 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkz56XN/build/debug/bin/kudu
/tmp/dist-test-taskkz56XN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127579900523-11321-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127579900523-11321-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127579900523-11321-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127579900523-11321-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.14.67:0
--local_ip_for_outbound_sockets=127.11.14.67
--webserver_interface=127.11.14.67
--webserver_port=0
--tserver_master_addrs=127.11.14.126:43679
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.67.keytab
--principal=kudu/127.11.14.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.14.84:45823
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127579900523-11321-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127579900523-11321-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127579900523-11321-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:33:46.396073 20526 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:33:46.396337 20526 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:33:46.396380 20526 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:33:46.400357 20526 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:33:46.400424 20526 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:33:46.400529 20526 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.14.67
I20260807 18:33:46.406216 20526 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.14.84:45823
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127579900523-11321-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127579900523-11321-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.67.keytab
--principal=kudu/127.11.14.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.14.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127579900523-11321-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.14.67
--webserver_port=0
--tserver_master_addrs=127.11.14.126:43679
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20526
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.14.67
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127579900523-11321-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:33:46.407347 20526 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:33:46.408377 20526 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:33:46.418283 20533 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:33:46.418363 20532 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:33:46.418298 20535 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:33:46.418565 20526 server_base.cc:1034] running on GCE node
I20260807 18:33:46.419006 20526 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:33:46.419672 20526 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:33:46.420876 20526 hybrid_clock.cc:648] HybridClock initialized: now 1786127626420870 us; error 40 us; skew 500 ppm
Aug 07 18:33:46 dist-test-slave-6k22 krb5kdc[20129](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127626, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:33:46.423840 20526 init.cc:377] Logged in from keytab as kudu/127.11.14.67@KRBTEST.COM (short username kudu)
I20260807 18:33:46.424712 20526 webserver.cc:460] Webserver started at http://127.11.14.67:38813/ using document root <none> and password file <none>
I20260807 18:33:46.425302 20526 fs_manager.cc:359] Metadata directory not provided
I20260807 18:33:46.425397 20526 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:33:46.425642 20526 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:33:46.427771 20526 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127579900523-11321-0/minicluster-data/ts-2/data/instance:
uuid: "bda02a409df54fa7a29c16794d0fc45c"
format_stamp: "Formatted at 2026-08-07 18:33:46 on dist-test-slave-6k22"
server_key: "05208c292badb099307b0c7985e10f60"
server_key_iv: "ed5024d4f307e0d301ffb1decc9e2733"
server_key_version: "encryptionkey@0"
I20260807 18:33:46.428372 20526 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127579900523-11321-0/minicluster-data/ts-2/wal/instance:
uuid: "bda02a409df54fa7a29c16794d0fc45c"
format_stamp: "Formatted at 2026-08-07 18:33:46 on dist-test-slave-6k22"
server_key: "05208c292badb099307b0c7985e10f60"
server_key_iv: "ed5024d4f307e0d301ffb1decc9e2733"
server_key_version: "encryptionkey@0"
I20260807 18:33:46.432101 20526 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:33:46.434621 20542 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:33:46.436000 20526 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 18:33:46.436136 20526 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127579900523-11321-0/minicluster-data/ts-2/data,/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127579900523-11321-0/minicluster-data/ts-2/wal
uuid: "bda02a409df54fa7a29c16794d0fc45c"
format_stamp: "Formatted at 2026-08-07 18:33:46 on dist-test-slave-6k22"
server_key: "05208c292badb099307b0c7985e10f60"
server_key_iv: "ed5024d4f307e0d301ffb1decc9e2733"
server_key_version: "encryptionkey@0"
I20260807 18:33:46.436244 20526 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127579900523-11321-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127579900523-11321-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127579900523-11321-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:33:46.483291 20526 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:33:46.486609 20526 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:33:46.486829 20526 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:33:46.487481 20526 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:33:46.488394 20526 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:33:46.488443 20526 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:33:46.488521 20526 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:33:46.488560 20526 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:33:46.498317 20526 rpc_server.cc:307] RPC server started. Bound to: 127.11.14.67:40779
I20260807 18:33:46.498366 20662 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.14.67:40779 every 8 connection(s)
I20260807 18:33:46.499277 20526 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127579900523-11321-0/minicluster-data/ts-2/data/info.pb
I20260807 18:33:46.503855 11321 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkz56XN/build/debug/bin/kudu as pid 20526
I20260807 18:33:46.504007 11321 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127579900523-11321-0/minicluster-data/ts-2/wal/instance
I20260807 18:33:46.504328 11321 external_mini_cluster.cc:1428] Setting key 2f0aa60301879ab31a512653afcb254a
Aug 07 18:33:46 dist-test-slave-6k22 krb5kdc[20129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127626, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.67@KRBTEST.COM for kudu/127.11.14.126@KRBTEST.COM
I20260807 18:33:46.513470 20218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:46.501216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.67:41707 (local address 127.11.14.126:43679)
0807 18:33:46.501360 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:46.501364 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:46.502291 (+ 927us) server_negotiation.cc:407] Connection header received
0807 18:33:46.503768 (+ 1477us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:46.503771 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:46.503819 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:46.503903 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:33:46.505464 (+ 1561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:46.506264 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:46.507633 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:46.507854 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:46.507938 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:46.510345 (+ 2407us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:33:46.510348 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:33:46.510350 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:33:46.510374 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:33:46.511751 (+ 1377us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:46.512207 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:46.512209 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:46.512210 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:46.512282 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:46.512566 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:46.512569 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:46.512570 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:46.512750 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:33:46.512824 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:46.513261 (+ 437us) server_negotiation.cc:299] Negotiation successful
0807 18:33:46.513353 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260807 18:33:46.514323 20665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:46.501595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:43679 (local address 127.11.14.67:41707)
0807 18:33:46.502091 (+ 496us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:46.502165 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:46.503481 (+ 1316us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:46.504112 (+ 631us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:46.504122 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:46.504608 (+ 486us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:33:46.505315 (+ 707us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:46.505337 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:46.506405 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:46.506408 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:46.507480 (+ 1072us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:46.507490 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:46.507987 (+ 497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:46.507990 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:46.508126 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:46.508743 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:33:46.508759 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:33:46.510189 (+ 1430us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:33:46.511853 (+ 1664us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:33:46.511858 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:33:46.511869 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:33:46.512122 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:33:46.512360 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:33:46.512362 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:33:46.512364 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:33:46.512489 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:33:46.512837 (+ 348us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:33:46.512842 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:33:46.513059 (+ 217us) client_negotiation.cc:769] Sending connection context
0807 18:33:46.513210 (+ 151us) client_negotiation.cc:240] Negotiation successful
0807 18:33:46.513468 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":303,"thread_start_us":148,"threads_started":1}
I20260807 18:33:46.515447 20663 heartbeater.cc:347] Connected to a master server at 127.11.14.126:43679
I20260807 18:33:46.515715 20663 heartbeater.cc:464] Registering TS with master...
I20260807 18:33:46.516206 20663 heartbeater.cc:511] Master 127.11.14.126:43679 requested a full tablet report, sending...
I20260807 18:33:46.517289 20173 ts_manager.cc:194] Registered new tserver with Master: bda02a409df54fa7a29c16794d0fc45c (127.11.14.67:40779)
I20260807 18:33:46.517935 20173 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.14.67@KRBTEST.COM'} at 127.11.14.67:41707
I20260807 18:33:46.518294 11321 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.11.14.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.14.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.14.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127579900523-11321-0/krb5kdc/oryx_127.11.14.65.keytab.
Entry for principal oryx/127.11.14.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127579900523-11321-0/krb5kdc/oryx_127.11.14.65.keytab.
WARNING: no policy specified for oryx/127.11.14.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.14.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.14.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127579900523-11321-0/krb5kdc/oryx_127.11.14.66.keytab.
Entry for principal oryx/127.11.14.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127579900523-11321-0/krb5kdc/oryx_127.11.14.66.keytab.
WARNING: no policy specified for oryx/127.11.14.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.14.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.14.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127579900523-11321-0/krb5kdc/oryx_127.11.14.67.keytab.
Entry for principal oryx/127.11.14.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127579900523-11321-0/krb5kdc/oryx_127.11.14.67.keytab.
I20260807 18:33:46.594556 11321 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkz56XN/build/debug/bin/kudu with pid 20243
I20260807 18:33:46.606117 11321 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkz56XN/build/debug/bin/kudu with pid 20386
I20260807 18:33:46.613142 11321 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkz56XN/build/debug/bin/kudu with pid 20526
I20260807 18:33:46.620504 11321 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkz56XN/build/debug/bin/kudu with pid 20143
I20260807 18:33:46.628196 11321 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkz56XN/build/debug/bin/kudu
/tmp/dist-test-taskkz56XN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127579900523-11321-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127579900523-11321-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127579900523-11321-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127579900523-11321-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.14.126:43679
--webserver_interface=127.11.14.126
--webserver_port=45961
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.126.keytab
--principal=kudu/127.11.14.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.14.84:45823
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.14.126:43679
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127579900523-11321-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127579900523-11321-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127579900523-11321-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:33:46.741541 20674 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:33:46.741791 20674 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:33:46.741842 20674 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:33:46.745884 20674 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:33:46.745941 20674 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:33:46.745960 20674 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:33:46.746001 20674 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:33:46.746037 20674 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:33:46.751273 20674 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.14.84:45823
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127579900523-11321-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127579900523-11321-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.14.126:43679
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.126.keytab
--principal=kudu/127.11.14.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.14.126:43679
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127579900523-11321-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.14.126
--webserver_port=45961
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20674
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127579900523-11321-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:33:46.752313 20674 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:33:46.753345 20674 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:33:46.761948 20681 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:33:46.761952 20680 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:33:46.762068 20674 server_base.cc:1034] running on GCE node
W20260807 18:33:46.761976 20683 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:33:46.762795 20674 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:33:46.764035 20674 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:33:46.765199 20674 hybrid_clock.cc:648] HybridClock initialized: now 1786127626765184 us; error 46 us; skew 500 ppm
Aug 07 18:33:46 dist-test-slave-6k22 krb5kdc[20129](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127626, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:33:46.768040 20674 init.cc:377] Logged in from keytab as kudu/127.11.14.126@KRBTEST.COM (short username kudu)
I20260807 18:33:46.768978 20674 webserver.cc:460] Webserver started at http://127.11.14.126:45961/ using document root <none> and password file <none>
I20260807 18:33:46.769565 20674 fs_manager.cc:359] Metadata directory not provided
I20260807 18:33:46.769685 20674 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:33:46.773573 20674 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 18:33:46.775738 20691 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:33:46.777192 20674 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 18:33:46.777344 20674 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127579900523-11321-0/minicluster-data/master-0/data,/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127579900523-11321-0/minicluster-data/master-0/wal
uuid: "15d605430db248bdbce86be96eb7d04a"
format_stamp: "Formatted at 2026-08-07 18:33:45 on dist-test-slave-6k22"
server_key: "60b90d58c888dcc52b0d48f9f3de33d7"
server_key_iv: "6299dc0e7cfbbadc6c1af995deca7220"
server_key_version: "encryptionkey@0"
I20260807 18:33:46.777882 20674 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127579900523-11321-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127579900523-11321-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127579900523-11321-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:33:46.792613 20674 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:33:46.795408 20674 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:33:46.795665 20674 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:33:46.802826 20674 rpc_server.cc:307] RPC server started. Bound to: 127.11.14.126:43679
I20260807 18:33:46.802839 20748 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.14.126:43679 every 8 connection(s)
I20260807 18:33:46.803972 20674 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127579900523-11321-0/minicluster-data/master-0/data/info.pb
I20260807 18:33:46.804983 11321 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkz56XN/build/debug/bin/kudu as pid 20674
I20260807 18:33:46.805534 11321 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkz56XN/build/debug/bin/kudu
/tmp/dist-test-taskkz56XN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127579900523-11321-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127579900523-11321-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127579900523-11321-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127579900523-11321-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.14.65:42313
--local_ip_for_outbound_sockets=127.11.14.65
--tserver_master_addrs=127.11.14.126:43679
--webserver_port=38617
--webserver_interface=127.11.14.65
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.65.keytab
--principal=kudu/127.11.14.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.14.84:45823
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.11.14.65
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127579900523-11321-0/krb5kdc/oryx_127.11.14.65.keytab with env {KRB5CCNAME=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127579900523-11321-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127579900523-11321-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127579900523-11321-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 18:33:46.809384 20749 sys_catalog.cc:263] Verifying existing consensus state
I20260807 18:33:46.811105 20749 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 15d605430db248bdbce86be96eb7d04a: Bootstrap starting.
I20260807 18:33:46.828420 20749 log.cc:824] T 00000000000000000000000000000000 P 15d605430db248bdbce86be96eb7d04a: Log is configured to *not* fsync() on all Append() calls
I20260807 18:33:46.835413 20749 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 15d605430db248bdbce86be96eb7d04a: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 18:33:46.835887 20749 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 15d605430db248bdbce86be96eb7d04a: Bootstrap complete.
I20260807 18:33:46.841442 20749 raft_consensus.cc:348] T 00000000000000000000000000000000 P 15d605430db248bdbce86be96eb7d04a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15d605430db248bdbce86be96eb7d04a" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 43679 } }
I20260807 18:33:46.841845 20749 raft_consensus.cc:729] T 00000000000000000000000000000000 P 15d605430db248bdbce86be96eb7d04a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 15d605430db248bdbce86be96eb7d04a, State: Initialized, Role: FOLLOWER
I20260807 18:33:46.842351 20749 consensus_queue.cc:260] T 00000000000000000000000000000000 P 15d605430db248bdbce86be96eb7d04a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15d605430db248bdbce86be96eb7d04a" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 43679 } }
I20260807 18:33:46.842473 20749 raft_consensus.cc:388] T 00000000000000000000000000000000 P 15d605430db248bdbce86be96eb7d04a [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:33:46.842564 20749 raft_consensus.cc:482] T 00000000000000000000000000000000 P 15d605430db248bdbce86be96eb7d04a [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:33:46.842664 20749 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 15d605430db248bdbce86be96eb7d04a [term 1 FOLLOWER]: Advancing to term 2
I20260807 18:33:46.844731 20749 raft_consensus.cc:504] T 00000000000000000000000000000000 P 15d605430db248bdbce86be96eb7d04a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15d605430db248bdbce86be96eb7d04a" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 43679 } }
I20260807 18:33:46.845098 20749 leader_election.cc:302] T 00000000000000000000000000000000 P 15d605430db248bdbce86be96eb7d04a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 15d605430db248bdbce86be96eb7d04a; no voters:
I20260807 18:33:46.845482 20749 leader_election.cc:288] T 00000000000000000000000000000000 P 15d605430db248bdbce86be96eb7d04a [CANDIDATE]: Term 2 election: Requested vote from peers
I20260807 18:33:46.845631 20753 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 15d605430db248bdbce86be96eb7d04a [term 2 FOLLOWER]: Leader election won for term 2
I20260807 18:33:46.845906 20753 raft_consensus.cc:686] T 00000000000000000000000000000000 P 15d605430db248bdbce86be96eb7d04a [term 2 LEADER]: Becoming Leader. State: Replica: 15d605430db248bdbce86be96eb7d04a, State: Running, Role: LEADER
I20260807 18:33:46.846295 20753 consensus_queue.cc:237] T 00000000000000000000000000000000 P 15d605430db248bdbce86be96eb7d04a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15d605430db248bdbce86be96eb7d04a" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 43679 } }
I20260807 18:33:46.846791 20749 sys_catalog.cc:564] T 00000000000000000000000000000000 P 15d605430db248bdbce86be96eb7d04a [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:33:46.847337 20753 sys_catalog.cc:455] T 00000000000000000000000000000000 P 15d605430db248bdbce86be96eb7d04a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 15d605430db248bdbce86be96eb7d04a. Latest consensus state: current_term: 2 leader_uuid: "15d605430db248bdbce86be96eb7d04a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15d605430db248bdbce86be96eb7d04a" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 43679 } } }
I20260807 18:33:46.847463 20753 sys_catalog.cc:458] T 00000000000000000000000000000000 P 15d605430db248bdbce86be96eb7d04a [sys.catalog]: This master's current role is: LEADER
I20260807 18:33:46.848109 20754 sys_catalog.cc:455] T 00000000000000000000000000000000 P 15d605430db248bdbce86be96eb7d04a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "15d605430db248bdbce86be96eb7d04a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15d605430db248bdbce86be96eb7d04a" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 43679 } } }
I20260807 18:33:46.848269 20754 sys_catalog.cc:458] T 00000000000000000000000000000000 P 15d605430db248bdbce86be96eb7d04a [sys.catalog]: This master's current role is: LEADER
I20260807 18:33:46.848924 20758 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:33:46.852308 20758 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:33:46.852948 20758 catalog_manager.cc:1259] Loaded cluster ID: 19583b40706a4986b2070159ee990882
I20260807 18:33:46.853014 20758 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:33:46.855367 20758 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:33:46.856065 20758 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 15d605430db248bdbce86be96eb7d04a: Loaded TSK: 0
I20260807 18:33:46.857313 20758 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 18:33:46.926499 20751 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:33:46.926838 20751 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:33:46.926919 20751 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:33:46.931389 20751 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:33:46.931624 20751 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:33:46.931845 20751 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.14.65
I20260807 18:33:46.938206 20751 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.14.84:45823
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127579900523-11321-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127579900523-11321-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127579900523-11321-0/krb5kdc/oryx_127.11.14.65.keytab
--principal=oryx/127.11.14.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.14.65:42313
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127579900523-11321-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.14.65
--webserver_port=38617
--tserver_master_addrs=127.11.14.126:43679
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20751
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.14.65
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127579900523-11321-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:33:46.939385 20751 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:33:46.940579 20751 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:33:46.952459 20779 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:33:46.952473 20778 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:33:46.952623 20781 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:33:46.953246 20751 server_base.cc:1034] running on GCE node
I20260807 18:33:46.953683 20751 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:33:46.954368 20751 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:33:46.955621 20751 hybrid_clock.cc:648] HybridClock initialized: now 1786127626955582 us; error 44 us; skew 500 ppm
Aug 07 18:33:46 dist-test-slave-6k22 krb5kdc[20129](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127626, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.14.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:33:46.958590 20751 init.cc:377] Logged in from keytab as oryx/127.11.14.65@KRBTEST.COM (short username oryx)
I20260807 18:33:46.959537 20751 webserver.cc:460] Webserver started at http://127.11.14.65:38617/ using document root <none> and password file <none>
I20260807 18:33:46.960189 20751 fs_manager.cc:359] Metadata directory not provided
I20260807 18:33:46.960310 20751 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:33:46.964455 20751 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.003s
I20260807 18:33:46.966745 20788 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:33:46.968168 20751 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 18:33:46.968312 20751 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127579900523-11321-0/minicluster-data/ts-0/data,/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127579900523-11321-0/minicluster-data/ts-0/wal
uuid: "efb72bd369324b0fa9a81ea56bb2ca2e"
format_stamp: "Formatted at 2026-08-07 18:33:45 on dist-test-slave-6k22"
server_key: "7232ee7d96b1457e65b652a9ab6552d1"
server_key_iv: "741b66dec5bcd1eea1b0d0d6ca2ac688"
server_key_version: "encryptionkey@0"
I20260807 18:33:46.968880 20751 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127579900523-11321-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127579900523-11321-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127579900523-11321-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:33:46.992336 20751 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:33:46.995780 20751 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:33:46.995992 20751 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:33:46.996620 20751 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:33:46.997686 20751 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:33:46.997761 20751 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:33:46.997844 20751 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:33:46.997886 20751 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:33:47.007314 20751 rpc_server.cc:307] RPC server started. Bound to: 127.11.14.65:42313
I20260807 18:33:47.007380 20915 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.14.65:42313 every 8 connection(s)
I20260807 18:33:47.008453 20751 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127579900523-11321-0/minicluster-data/ts-0/data/info.pb
I20260807 18:33:47.013602 11321 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkz56XN/build/debug/bin/kudu as pid 20751
I20260807 18:33:47.014089 11321 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkz56XN/build/debug/bin/kudu
/tmp/dist-test-taskkz56XN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127579900523-11321-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127579900523-11321-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127579900523-11321-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127579900523-11321-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.14.66:42163
--local_ip_for_outbound_sockets=127.11.14.66
--tserver_master_addrs=127.11.14.126:43679
--webserver_port=33585
--webserver_interface=127.11.14.66
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.66.keytab
--principal=kudu/127.11.14.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.14.84:45823
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.11.14.66
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127579900523-11321-0/krb5kdc/oryx_127.11.14.66.keytab with env {KRB5CCNAME=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127579900523-11321-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127579900523-11321-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127579900523-11321-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 18:33:47 dist-test-slave-6k22 krb5kdc[20129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.14.65@KRBTEST.COM for oryx/127.11.14.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:33:47 dist-test-slave-6k22 krb5kdc[20129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.14.65@KRBTEST.COM for oryx/127.11.14.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:33:47.022367 20920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:47.010666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:43679 (local address 127.11.14.65:40171)
0807 18:33:47.011167 (+ 501us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:47.011211 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:47.012511 (+ 1300us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:47.013608 (+ 1097us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:47.013616 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:47.014065 (+ 449us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:33:47.014635 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:47.014651 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:47.016249 (+ 1598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:47.016252 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:47.017110 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:47.017121 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:47.018204 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:47.018207 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:47.018364 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:47.018989 (+ 625us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:33:47.019006 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:33:47.021616 (+ 2610us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.14.126:43679: Server oryx/127.11.14.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":302,"thread_start_us":152,"threads_started":1}
W20260807 18:33:47.022900 20916 heartbeater.cc:644] Failed to heartbeat to 127.11.14.126:43679 (0 consecutive failures): Not authorized: Failed to ping master at 127.11.14.126:43679: Client connection negotiation failed: client connection to 127.11.14.126:43679: Server oryx/127.11.14.126@KRBTEST.COM not found in Kerberos database
W20260807 18:33:47.023433 20918 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:33:47.010600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.65:40171 (local address 127.11.14.126:43679)
0807 18:33:47.011085 (+ 485us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:47.011107 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:47.011331 (+ 224us) server_negotiation.cc:407] Connection header received
0807 18:33:47.012893 (+ 1562us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:47.012904 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:47.013193 (+ 289us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:47.013537 (+ 344us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:33:47.014833 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:47.016093 (+ 1260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:47.017245 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:47.018040 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:47.018238 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:47.021929 (+ 3691us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.14.65:40171: BlockingRecv error: recv got EOF from 127.11.14.65:40171 (error 108)
Metrics: {"server-negotiator.queue_time_us":321,"thread_start_us":100,"threads_started":1}
Aug 07 18:33:47 dist-test-slave-6k22 krb5kdc[20129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.14.65@KRBTEST.COM for oryx/127.11.14.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:33:47 dist-test-slave-6k22 krb5kdc[20129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.14.65@KRBTEST.COM for oryx/127.11.14.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:33:47.029361 20920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:47.023419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:43679 (local address 127.11.14.65:48501)
0807 18:33:47.023647 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:47.023666 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:47.023752 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:47.024075 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:47.024080 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:47.024268 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:33:47.024420 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:47.024430 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:47.025168 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:47.025174 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:47.026186 (+ 1012us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:47.026204 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:47.026663 (+ 459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:47.026668 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:47.026789 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:47.027302 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:33:47.027318 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:33:47.029207 (+ 1889us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.14.126:43679: Server oryx/127.11.14.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":121}
W20260807 18:33:47.029449 20922 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:33:47.023518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.65:48501 (local address 127.11.14.126:43679)
0807 18:33:47.023822 (+ 304us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:47.023834 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:47.023850 (+ 16us) server_negotiation.cc:407] Connection header received
0807 18:33:47.023887 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:47.023890 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:47.023933 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:47.024033 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:33:47.024550 (+ 517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:47.025045 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:47.026324 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:47.026544 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:47.026619 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:47.029323 (+ 2704us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.14.65:48501: BlockingRecv error: recv got EOF from 127.11.14.65:48501 (error 108)
Metrics: {"server-negotiator.queue_time_us":239,"thread_start_us":125,"threads_started":1}
Aug 07 18:33:47 dist-test-slave-6k22 krb5kdc[20129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.14.65@KRBTEST.COM for oryx/127.11.14.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:33:47 dist-test-slave-6k22 krb5kdc[20129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.14.65@KRBTEST.COM for oryx/127.11.14.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:33:47.034978 20920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:47.029957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:43679 (local address 127.11.14.65:54997)
0807 18:33:47.030077 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:47.030093 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:47.030171 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:47.030409 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:47.030411 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:47.030528 (+ 117us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:33:47.030594 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:47.030599 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:47.031628 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:47.031631 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:47.032121 (+ 490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:47.032128 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:47.032685 (+ 557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:47.032688 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:47.032742 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:47.033196 (+ 454us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:33:47.033208 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:33:47.034862 (+ 1654us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.14.126:43679: Server oryx/127.11.14.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":47}
W20260807 18:33:47.035157 20916 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 18:33:47.035308 20922 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:33:47.030047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.65:54997 (local address 127.11.14.126:43679)
0807 18:33:47.030177 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:47.030181 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:47.030222 (+ 41us) server_negotiation.cc:407] Connection header received
0807 18:33:47.030256 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:47.030259 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:47.030310 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:47.030384 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:33:47.030695 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:47.031470 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:47.032225 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:47.032577 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:47.032678 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:47.034981 (+ 2303us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.14.65:54997: BlockingRecv error: recv got EOF from 127.11.14.65:54997 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
W20260807 18:33:47.134532 20921 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:33:47.134814 20921 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:33:47.134900 20921 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:33:47.139009 20921 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:33:47.139076 20921 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:33:47.139182 20921 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.14.66
I20260807 18:33:47.144549 20921 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.14.84:45823
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127579900523-11321-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127579900523-11321-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127579900523-11321-0/krb5kdc/oryx_127.11.14.66.keytab
--principal=oryx/127.11.14.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.14.66:42163
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127579900523-11321-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.14.66
--webserver_port=33585
--tserver_master_addrs=127.11.14.126:43679
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20921
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.14.66
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127579900523-11321-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:33:47.145614 20921 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:33:47.146787 20921 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:33:47.156605 20928 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:33:47.156734 20929 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:33:47.156612 20931 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:33:47.156832 20921 server_base.cc:1034] running on GCE node
I20260807 18:33:47.157423 20921 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:33:47.158010 20921 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:33:47.159180 20921 hybrid_clock.cc:648] HybridClock initialized: now 1786127627159165 us; error 25 us; skew 500 ppm
Aug 07 18:33:47 dist-test-slave-6k22 krb5kdc[20129](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127627, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.14.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:33:47.162310 20921 init.cc:377] Logged in from keytab as oryx/127.11.14.66@KRBTEST.COM (short username oryx)
I20260807 18:33:47.163223 20921 webserver.cc:460] Webserver started at http://127.11.14.66:33585/ using document root <none> and password file <none>
I20260807 18:33:47.163853 20921 fs_manager.cc:359] Metadata directory not provided
I20260807 18:33:47.163937 20921 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:33:47.167927 20921 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260807 18:33:47.170004 20938 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:33:47.171270 20921 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 18:33:47.171427 20921 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127579900523-11321-0/minicluster-data/ts-1/data,/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127579900523-11321-0/minicluster-data/ts-1/wal
uuid: "f84e9ea34d3a47d8952e0e84104765c5"
format_stamp: "Formatted at 2026-08-07 18:33:46 on dist-test-slave-6k22"
server_key: "b2cf9757edfbf58995b9313bf5d59a33"
server_key_iv: "a9eb0082a48e55c25f3eff1ad2244974"
server_key_version: "encryptionkey@0"
I20260807 18:33:47.172122 20921 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127579900523-11321-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127579900523-11321-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127579900523-11321-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:33:47.188110 20921 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:33:47.191478 20921 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:33:47.191844 20921 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:33:47.192655 20921 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:33:47.193576 20921 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:33:47.193639 20921 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:33:47.193683 20921 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:33:47.193707 20921 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:33:47.203074 20921 rpc_server.cc:307] RPC server started. Bound to: 127.11.14.66:42163
I20260807 18:33:47.203101 21060 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.14.66:42163 every 8 connection(s)
I20260807 18:33:47.204186 20921 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127579900523-11321-0/minicluster-data/ts-1/data/info.pb
I20260807 18:33:47.212162 11321 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkz56XN/build/debug/bin/kudu as pid 20921
I20260807 18:33:47.212762 11321 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkz56XN/build/debug/bin/kudu
/tmp/dist-test-taskkz56XN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127579900523-11321-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127579900523-11321-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127579900523-11321-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127579900523-11321-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.14.67:40779
--local_ip_for_outbound_sockets=127.11.14.67
--tserver_master_addrs=127.11.14.126:43679
--webserver_port=38813
--webserver_interface=127.11.14.67
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.67.keytab
--principal=kudu/127.11.14.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.14.84:45823
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.11.14.67
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127579900523-11321-0/krb5kdc/oryx_127.11.14.67.keytab with env {KRB5CCNAME=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127579900523-11321-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127579900523-11321-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127579900523-11321-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 18:33:47 dist-test-slave-6k22 krb5kdc[20129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.14.66@KRBTEST.COM for oryx/127.11.14.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:33:47 dist-test-slave-6k22 krb5kdc[20129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.14.66@KRBTEST.COM for oryx/127.11.14.126@KRBTEST.COM, Server not found in Kerberos database
W20260807 18:33:47.215541 20922 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:33:47.206144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.66:50935 (local address 127.11.14.126:43679)
0807 18:33:47.206302 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:47.206305 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:47.207024 (+ 719us) server_negotiation.cc:407] Connection header received
0807 18:33:47.208415 (+ 1391us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:47.208418 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:47.208467 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:47.208531 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:33:47.209679 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:47.210212 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:47.211302 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:47.211507 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:47.211633 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:47.215364 (+ 3731us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.14.66:50935: BlockingRecv error: recv got EOF from 127.11.14.66:50935 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
I20260807 18:33:47.216118 21063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:47.206438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:43679 (local address 127.11.14.66:50935)
0807 18:33:47.206866 (+ 428us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:47.206909 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:47.208194 (+ 1285us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:47.208703 (+ 509us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:47.208710 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:47.209050 (+ 340us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:33:47.209548 (+ 498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:47.209561 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:47.210324 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:47.210326 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:47.211194 (+ 868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:47.211202 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:47.211689 (+ 487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:47.211691 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:47.211957 (+ 266us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:47.212597 (+ 640us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:33:47.212618 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:33:47.215178 (+ 2560us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.14.126:43679: Server oryx/127.11.14.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":104,"threads_started":1}
W20260807 18:33:47.216665 21061 heartbeater.cc:644] Failed to heartbeat to 127.11.14.126:43679 (0 consecutive failures): Not authorized: Failed to ping master at 127.11.14.126:43679: Client connection negotiation failed: client connection to 127.11.14.126:43679: Server oryx/127.11.14.126@KRBTEST.COM not found in Kerberos database
Aug 07 18:33:47 dist-test-slave-6k22 krb5kdc[20129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.14.66@KRBTEST.COM for oryx/127.11.14.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:33:47 dist-test-slave-6k22 krb5kdc[20129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.14.66@KRBTEST.COM for oryx/127.11.14.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:33:47.222693 21063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:47.217141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:43679 (local address 127.11.14.66:53597)
0807 18:33:47.217274 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:47.217298 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:47.217376 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:47.217635 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:47.217637 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:47.217783 (+ 146us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:33:47.217931 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:47.217937 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:47.219052 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:47.219054 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:47.219640 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:47.219649 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:47.220293 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:47.220295 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:47.220361 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:47.220858 (+ 497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:33:47.220870 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:33:47.222608 (+ 1738us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.14.126:43679: Server oryx/127.11.14.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":67}
W20260807 18:33:47.222963 20922 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:33:47.217229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.66:53597 (local address 127.11.14.126:43679)
0807 18:33:47.217363 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:47.217369 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:47.217386 (+ 17us) server_negotiation.cc:407] Connection header received
0807 18:33:47.217439 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:47.217445 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:47.217510 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:47.217609 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:33:47.218051 (+ 442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:47.218938 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:47.219764 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:47.220168 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:47.220307 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:47.222801 (+ 2494us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.14.66:53597: BlockingRecv error: recv got EOF from 127.11.14.66:53597 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 07 18:33:47 dist-test-slave-6k22 krb5kdc[20129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.14.66@KRBTEST.COM for oryx/127.11.14.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:33:47 dist-test-slave-6k22 krb5kdc[20129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.14.66@KRBTEST.COM for oryx/127.11.14.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:33:47.228681 21063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:47.223339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:43679 (local address 127.11.14.66:56603)
0807 18:33:47.223429 (+ 90us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:47.223445 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:47.223526 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:47.223866 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:47.223868 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:47.223990 (+ 122us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:33:47.224058 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:47.224062 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:47.225145 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:47.225147 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:47.225648 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:47.225655 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:47.226236 (+ 581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:47.226239 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:47.226295 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:47.226737 (+ 442us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:33:47.226748 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:33:47.228564 (+ 1816us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.14.126:43679: Server oryx/127.11.14.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":27}
W20260807 18:33:47.228801 20922 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:33:47.223403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.66:56603 (local address 127.11.14.126:43679)
0807 18:33:47.223546 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:47.223551 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:47.223589 (+ 38us) server_negotiation.cc:407] Connection header received
0807 18:33:47.223672 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:47.223677 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:47.223729 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:47.223822 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:33:47.224169 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:47.225039 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:47.225788 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:47.226127 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:47.226230 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:47.228676 (+ 2446us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.14.66:56603: BlockingRecv error: recv got EOF from 127.11.14.66:56603 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
W20260807 18:33:47.229161 21061 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 18:33:47.329461 21064 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:33:47.329809 21064 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:33:47.329887 21064 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:33:47.333958 21064 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:33:47.334053 21064 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:33:47.334144 21064 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.14.67
I20260807 18:33:47.339406 21064 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.14.84:45823
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127579900523-11321-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127579900523-11321-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127579900523-11321-0/krb5kdc/oryx_127.11.14.67.keytab
--principal=oryx/127.11.14.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.14.67:40779
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127579900523-11321-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.14.67
--webserver_port=38813
--tserver_master_addrs=127.11.14.126:43679
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21064
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.14.67
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127579900523-11321-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:33:47.340495 21064 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:33:47.341531 21064 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:33:47.351608 21073 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:33:47.351583 21069 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:33:47.351662 21064 server_base.cc:1034] running on GCE node
W20260807 18:33:47.351634 21070 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:33:47.352257 21064 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:33:47.352991 21064 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:33:47.354180 21064 hybrid_clock.cc:648] HybridClock initialized: now 1786127627354152 us; error 46 us; skew 500 ppm
Aug 07 18:33:47 dist-test-slave-6k22 krb5kdc[20129](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127627, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.14.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:33:47.356914 21064 init.cc:377] Logged in from keytab as oryx/127.11.14.67@KRBTEST.COM (short username oryx)
I20260807 18:33:47.357760 21064 webserver.cc:460] Webserver started at http://127.11.14.67:38813/ using document root <none> and password file <none>
I20260807 18:33:47.358378 21064 fs_manager.cc:359] Metadata directory not provided
I20260807 18:33:47.358457 21064 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:33:47.362262 21064 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.001s
I20260807 18:33:47.364365 21080 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:33:47.365588 21064 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260807 18:33:47.365684 21064 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127579900523-11321-0/minicluster-data/ts-2/data,/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127579900523-11321-0/minicluster-data/ts-2/wal
uuid: "bda02a409df54fa7a29c16794d0fc45c"
format_stamp: "Formatted at 2026-08-07 18:33:46 on dist-test-slave-6k22"
server_key: "05208c292badb099307b0c7985e10f60"
server_key_iv: "ed5024d4f307e0d301ffb1decc9e2733"
server_key_version: "encryptionkey@0"
I20260807 18:33:47.366211 21064 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127579900523-11321-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127579900523-11321-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127579900523-11321-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:33:47.399770 21064 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:33:47.402721 21064 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:33:47.402895 21064 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:33:47.403468 21064 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:33:47.404326 21064 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:33:47.404374 21064 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:33:47.404417 21064 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:33:47.404446 21064 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:33:47.413733 21064 rpc_server.cc:307] RPC server started. Bound to: 127.11.14.67:40779
I20260807 18:33:47.413791 21202 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.14.67:40779 every 8 connection(s)
I20260807 18:33:47.415127 21064 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786127579900523-11321-0/minicluster-data/ts-2/data/info.pb
I20260807 18:33:47.421070 11321 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkz56XN/build/debug/bin/kudu as pid 21064
Aug 07 18:33:47 dist-test-slave-6k22 krb5kdc[20129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.14.67@KRBTEST.COM for oryx/127.11.14.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:33:47 dist-test-slave-6k22 krb5kdc[20129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.14.67@KRBTEST.COM for oryx/127.11.14.126@KRBTEST.COM, Server not found in Kerberos database
W20260807 18:33:47.427551 20922 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:33:47.416950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.67:35863 (local address 127.11.14.126:43679)
0807 18:33:47.417093 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:47.417096 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:47.417908 (+ 812us) server_negotiation.cc:407] Connection header received
0807 18:33:47.419241 (+ 1333us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:47.419244 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:47.419305 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:47.419409 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:33:47.420671 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:47.421332 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:47.422910 (+ 1578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:47.423169 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:47.423346 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:47.427406 (+ 4060us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.14.67:35863: BlockingRecv error: recv got EOF from 127.11.14.67:35863 (error 108)
Metrics: {"mutex_wait_us":26,"server-negotiator.queue_time_us":52}
I20260807 18:33:47.428223 21205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:47.417301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:43679 (local address 127.11.14.67:35863)
0807 18:33:47.417747 (+ 446us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:47.417790 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:47.419020 (+ 1230us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:47.419652 (+ 632us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:47.419660 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:47.420027 (+ 367us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:33:47.420522 (+ 495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:47.420536 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:47.421464 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:47.421474 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:47.422753 (+ 1279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:47.422763 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:47.423371 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:47.423375 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:47.423527 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:47.424252 (+ 725us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:33:47.424275 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:33:47.427250 (+ 2975us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.14.126:43679: Server oryx/127.11.14.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":119,"threads_started":1}
W20260807 18:33:47.428979 21203 heartbeater.cc:644] Failed to heartbeat to 127.11.14.126:43679 (0 consecutive failures): Not authorized: Failed to ping master at 127.11.14.126:43679: Client connection negotiation failed: client connection to 127.11.14.126:43679: Server oryx/127.11.14.126@KRBTEST.COM not found in Kerberos database
I20260807 18:33:47.431900 20918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:47.422177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40472 (local address 127.11.14.126:43679)
0807 18:33:47.422330 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:47.422334 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:47.422348 (+ 14us) server_negotiation.cc:407] Connection header received
0807 18:33:47.422382 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:47.422384 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:47.422430 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:47.422515 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:33:47.423277 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:47.423998 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:47.425254 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:47.425606 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:47.425688 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:47.427406 (+ 1718us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:33:47.427413 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:33:47.427433 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:33:47.427463 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:33:47.429563 (+ 2100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:47.430103 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:47.430115 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:47.430120 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:47.430236 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:33:47.430545 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:33:47.430548 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:33:47.430550 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:33:47.430983 (+ 433us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:33:47.431182 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0807 18:33:47.431463 (+ 281us) server_negotiation.cc:299] Negotiation successful
0807 18:33:47.431773 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":21,"server-negotiator.queue_time_us":68}
Aug 07 18:33:47 dist-test-slave-6k22 krb5kdc[20129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.14.67@KRBTEST.COM for oryx/127.11.14.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:33:47 dist-test-slave-6k22 krb5kdc[20129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.14.67@KRBTEST.COM for oryx/127.11.14.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:33:47.435101 21205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:47.429420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:43679 (local address 127.11.14.67:59203)
0807 18:33:47.429688 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:47.429703 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:47.429806 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:47.430034 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:47.430040 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:47.430189 (+ 149us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:33:47.430310 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:47.430319 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:47.431233 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:47.431237 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:47.431966 (+ 729us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:47.431977 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:47.432421 (+ 444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:47.432425 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:47.432515 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:47.432957 (+ 442us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:33:47.432971 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:33:47.434961 (+ 1990us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.14.126:43679: Server oryx/127.11.14.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":180}
W20260807 18:33:47.435220 20922 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:33:47.429540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.67:59203 (local address 127.11.14.126:43679)
0807 18:33:47.429679 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:47.429685 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:47.429771 (+ 86us) server_negotiation.cc:407] Connection header received
0807 18:33:47.429888 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:47.429890 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:47.429924 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:47.429984 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:33:47.430419 (+ 435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:47.431129 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:47.432105 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:47.432303 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:47.432390 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:47.435098 (+ 2708us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.14.67:59203: BlockingRecv error: recv got EOF from 127.11.14.67:59203 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Aug 07 18:33:47 dist-test-slave-6k22 krb5kdc[20129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.14.67@KRBTEST.COM for oryx/127.11.14.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:33:47 dist-test-slave-6k22 krb5kdc[20129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.14.67@KRBTEST.COM for oryx/127.11.14.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:33:47.439915 21205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:47.435687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:43679 (local address 127.11.14.67:47165)
0807 18:33:47.435830 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:47.435845 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:47.435920 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:47.436081 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:47.436083 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:47.436201 (+ 118us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:33:47.436266 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:47.436270 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:47.436931 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:47.436934 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:47.437430 (+ 496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:47.437437 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:47.437829 (+ 392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:47.437831 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:47.437887 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:47.438241 (+ 354us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:33:47.438252 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:33:47.439828 (+ 1576us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.14.126:43679: Server oryx/127.11.14.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":56}
W20260807 18:33:47.440055 20922 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:33:47.435752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.67:47165 (local address 127.11.14.126:43679)
0807 18:33:47.435906 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:47.435909 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:47.435921 (+ 12us) server_negotiation.cc:407] Connection header received
0807 18:33:47.435980 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:47.435981 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:47.436012 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:47.436074 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:33:47.436361 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:47.436829 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:47.437527 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:47.437726 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:47.437802 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:47.439954 (+ 2152us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.14.67:47165: BlockingRecv error: recv got EOF from 127.11.14.67:47165 (error 108)
Metrics: {"server-negotiator.queue_time_us":92}
W20260807 18:33:47.440140 21203 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 07 18:33:48 dist-test-slave-6k22 krb5kdc[20129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.14.65@KRBTEST.COM for oryx/127.11.14.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:33:48 dist-test-slave-6k22 krb5kdc[20129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.14.65@KRBTEST.COM for oryx/127.11.14.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:33:48.041455 21207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:48.035963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:43679 (local address 127.11.14.65:50603)
0807 18:33:48.036287 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:48.036304 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:48.036452 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:48.036785 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:48.036788 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:48.036973 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:33:48.037075 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:48.037081 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:48.037797 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:48.037799 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:48.038378 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:48.038385 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:48.038795 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:48.038797 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:48.038859 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:48.039425 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:33:48.039446 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:33:48.041333 (+ 1887us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.14.126:43679: Server oryx/127.11.14.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":96,"threads_started":1}
W20260807 18:33:48.041595 21208 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:33:48.036027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.65:50603 (local address 127.11.14.126:43679)
0807 18:33:48.036324 (+ 297us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:48.036329 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:48.036379 (+ 50us) server_negotiation.cc:407] Connection header received
0807 18:33:48.036571 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:48.036577 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:48.036640 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:48.036734 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:33:48.037179 (+ 445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:48.037703 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:48.038481 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:48.038696 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:48.038772 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:48.041461 (+ 2689us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.14.65:50603: BlockingRecv error: recv got EOF from 127.11.14.65:50603 (error 108)
Metrics: {"server-negotiator.queue_time_us":198,"thread_start_us":98,"threads_started":1}
Aug 07 18:33:48 dist-test-slave-6k22 krb5kdc[20129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.14.66@KRBTEST.COM for oryx/127.11.14.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:33:48 dist-test-slave-6k22 krb5kdc[20129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.14.66@KRBTEST.COM for oryx/127.11.14.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:33:48.235697 21209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:48.229796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:43679 (local address 127.11.14.66:43337)
0807 18:33:48.230066 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:48.230079 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:48.230179 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:48.230426 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:48.230428 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:48.230610 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:33:48.230747 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:48.230752 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:48.231706 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:48.231710 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:48.232555 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:48.232565 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:48.232990 (+ 425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:48.232993 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:48.233063 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:48.233623 (+ 560us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:33:48.233635 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:33:48.235507 (+ 1872us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.14.126:43679: Server oryx/127.11.14.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":73,"threads_started":1}
W20260807 18:33:48.235762 21208 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:33:48.229950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.66:43337 (local address 127.11.14.126:43679)
0807 18:33:48.230115 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:48.230119 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:48.230133 (+ 14us) server_negotiation.cc:407] Connection header received
0807 18:33:48.230268 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:48.230270 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:48.230316 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:48.230386 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:33:48.230854 (+ 468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:48.231559 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:48.232663 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:48.232888 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:48.232969 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:48.235654 (+ 2685us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.14.66:43337: BlockingRecv error: recv got EOF from 127.11.14.66:43337 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
Aug 07 18:33:48 dist-test-slave-6k22 krb5kdc[20129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.14.67@KRBTEST.COM for oryx/127.11.14.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:33:48 dist-test-slave-6k22 krb5kdc[20129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.14.67@KRBTEST.COM for oryx/127.11.14.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:33:48.446844 21210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:48.440943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:43679 (local address 127.11.14.67:40913)
0807 18:33:48.441266 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:48.441282 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:48.441394 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:48.441716 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:48.441719 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:48.441936 (+ 217us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:33:48.442045 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:48.442051 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:48.442939 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:48.442944 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:48.443692 (+ 748us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:48.443702 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:48.444226 (+ 524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:48.444229 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:48.444321 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:48.444819 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:33:48.444830 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:33:48.446641 (+ 1811us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.14.126:43679: Server oryx/127.11.14.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":74,"threads_started":1}
W20260807 18:33:48.446959 21208 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:33:48.441108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.67:40913 (local address 127.11.14.126:43679)
0807 18:33:48.441431 (+ 323us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:48.441437 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:48.441455 (+ 18us) server_negotiation.cc:407] Connection header received
0807 18:33:48.441503 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:48.441506 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:48.441567 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:48.441687 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:33:48.442186 (+ 499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:48.442814 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:48.443881 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:48.444102 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:48.444210 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:48.446843 (+ 2633us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.14.67:40913: BlockingRecv error: recv got EOF from 127.11.14.67:40913 (error 108)
Metrics: {"server-negotiator.queue_time_us":225}
Aug 07 18:33:49 dist-test-slave-6k22 krb5kdc[20129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.14.65@KRBTEST.COM for oryx/127.11.14.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:33:49 dist-test-slave-6k22 krb5kdc[20129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.14.65@KRBTEST.COM for oryx/127.11.14.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:33:49.048451 21211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:49.042380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:43679 (local address 127.11.14.65:44459)
0807 18:33:49.042698 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:49.042715 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:49.042827 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:49.043144 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:49.043147 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:49.043395 (+ 248us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:33:49.043541 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:49.043549 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:49.044529 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:49.044531 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:49.045096 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:49.045105 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:49.045534 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:49.045536 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:49.045597 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:49.046071 (+ 474us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:33:49.046083 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:33:49.048251 (+ 2168us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.14.126:43679: Server oryx/127.11.14.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":106,"threads_started":1}
W20260807 18:33:49.048492 21212 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:33:49.042531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.65:44459 (local address 127.11.14.126:43679)
0807 18:33:49.042799 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:49.042803 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:49.042818 (+ 15us) server_negotiation.cc:407] Connection header received
0807 18:33:49.042936 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:49.042939 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:49.042992 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:49.043094 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:33:49.043721 (+ 627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:49.044415 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:49.045213 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:49.045430 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:49.045512 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:49.048375 (+ 2863us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.14.65:44459: BlockingRecv error: recv got EOF from 127.11.14.65:44459 (error 108)
Metrics: {"server-negotiator.queue_time_us":190,"thread_start_us":90,"threads_started":1}
Aug 07 18:33:49 dist-test-slave-6k22 krb5kdc[20129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.14.66@KRBTEST.COM for oryx/127.11.14.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:33:49 dist-test-slave-6k22 krb5kdc[20129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.14.66@KRBTEST.COM for oryx/127.11.14.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:33:49.249708 21213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:49.236570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:43679 (local address 127.11.14.66:41775)
0807 18:33:49.236870 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:49.236883 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:49.237020 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:49.237296 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:49.237299 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:49.237512 (+ 213us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:33:49.237609 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:49.237615 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:49.238404 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:49.238410 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:49.238978 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:49.238986 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:49.239460 (+ 474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:49.239464 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:49.239550 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:49.247042 (+ 7492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:33:49.247067 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:33:49.249521 (+ 2454us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.14.126:43679: Server oryx/127.11.14.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":85,"threads_started":1}
W20260807 18:33:49.249905 21212 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:33:49.236641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.66:41775 (local address 127.11.14.126:43679)
0807 18:33:49.236786 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:49.236791 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:49.236959 (+ 168us) server_negotiation.cc:407] Connection header received
0807 18:33:49.237124 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:49.237128 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:49.237181 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:49.237270 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:33:49.237731 (+ 461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:49.238273 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:49.239105 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:49.239329 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:49.239445 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:49.249668 (+ 10223us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.14.66:41775: BlockingRecv error: recv got EOF from 127.11.14.66:41775 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 07 18:33:49 dist-test-slave-6k22 krb5kdc[20129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.14.67@KRBTEST.COM for oryx/127.11.14.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:33:49 dist-test-slave-6k22 krb5kdc[20129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.14.67@KRBTEST.COM for oryx/127.11.14.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:33:49.453850 21214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:49.447753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:43679 (local address 127.11.14.67:47053)
0807 18:33:49.448106 (+ 353us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:49.448123 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:49.448271 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:49.448546 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:49.448549 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:49.448804 (+ 255us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:33:49.448928 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:49.448935 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:49.449711 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:49.449716 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:49.450399 (+ 683us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:49.450409 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:49.450923 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:49.450928 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:49.451077 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:49.451689 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:33:49.451702 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:33:49.453654 (+ 1952us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.14.126:43679: Server oryx/127.11.14.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":94,"threads_started":1}
W20260807 18:33:49.453888 21212 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:33:49.447906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.67:47053 (local address 127.11.14.126:43679)
0807 18:33:49.448061 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:49.448068 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:49.448220 (+ 152us) server_negotiation.cc:407] Connection header received
0807 18:33:49.448389 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:49.448394 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:49.448450 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:49.448524 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:33:49.449060 (+ 536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:49.449589 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:49.450526 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:49.450760 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:49.450898 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:49.453775 (+ 2877us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.14.67:47053: BlockingRecv error: recv got EOF from 127.11.14.67:47053 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 07 18:33:50 dist-test-slave-6k22 krb5kdc[20129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.14.65@KRBTEST.COM for oryx/127.11.14.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:33:50 dist-test-slave-6k22 krb5kdc[20129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.14.65@KRBTEST.COM for oryx/127.11.14.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:33:50.054692 21215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:50.049303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:43679 (local address 127.11.14.65:33485)
0807 18:33:50.049575 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:50.049589 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:50.049680 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:50.049932 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:50.049934 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:50.050117 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:33:50.050220 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:50.050225 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:50.050944 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:50.050947 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:50.051651 (+ 704us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:50.051662 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:50.052097 (+ 435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:50.052100 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:50.052161 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:50.052665 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:33:50.052677 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:33:50.054573 (+ 1896us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.14.126:43679: Server oryx/127.11.14.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":95,"threads_started":1}
W20260807 18:33:50.054860 21216 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:33:50.049385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.65:33485 (local address 127.11.14.126:43679)
0807 18:33:50.049615 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:50.049618 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:50.049633 (+ 15us) server_negotiation.cc:407] Connection header received
0807 18:33:50.049763 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:50.049766 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:50.049810 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:50.049891 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:33:50.050316 (+ 425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:50.050847 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:50.051765 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:50.051992 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:50.052077 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:50.054713 (+ 2636us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.14.65:33485: BlockingRecv error: recv got EOF from 127.11.14.65:33485 (error 108)
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":85,"threads_started":1}
Aug 07 18:33:50 dist-test-slave-6k22 krb5kdc[20129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.14.66@KRBTEST.COM for oryx/127.11.14.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:33:50 dist-test-slave-6k22 krb5kdc[20129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.14.66@KRBTEST.COM for oryx/127.11.14.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:33:50.256476 21217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:50.250670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:43679 (local address 127.11.14.66:42299)
0807 18:33:50.250978 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:50.250997 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:50.251160 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:50.251440 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:50.251443 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:50.251683 (+ 240us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:33:50.251840 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:50.251847 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:50.252658 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:50.252663 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:50.253243 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:50.253252 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:50.253721 (+ 469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:50.253724 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:50.253805 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:50.254336 (+ 531us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:33:50.254348 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:33:50.256288 (+ 1940us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.14.126:43679: Server oryx/127.11.14.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":93,"threads_started":1}
W20260807 18:33:50.256507 21216 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:33:50.250759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.66:42299 (local address 127.11.14.126:43679)
0807 18:33:50.250931 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:50.250936 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:50.251098 (+ 162us) server_negotiation.cc:407] Connection header received
0807 18:33:50.251266 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:50.251269 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:50.251322 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:50.251412 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:33:50.251965 (+ 553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:50.252550 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:50.253382 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:50.253604 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:50.253715 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:50.256404 (+ 2689us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.14.66:42299: BlockingRecv error: recv got EOF from 127.11.14.66:42299 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Aug 07 18:33:50 dist-test-slave-6k22 krb5kdc[20129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.14.67@KRBTEST.COM for oryx/127.11.14.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:33:50 dist-test-slave-6k22 krb5kdc[20129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.14.67@KRBTEST.COM for oryx/127.11.14.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:33:50.460712 21218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:50.454775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:43679 (local address 127.11.14.67:39949)
0807 18:33:50.455086 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:50.455101 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:50.455206 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:50.455450 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:50.455452 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:50.455673 (+ 221us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:33:50.455775 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:50.455780 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:50.456558 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:50.456560 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:50.457162 (+ 602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:50.457171 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:50.457720 (+ 549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:50.457724 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:50.457822 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:50.458494 (+ 672us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:33:50.458510 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:33:50.460526 (+ 2016us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.14.126:43679: Server oryx/127.11.14.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":118,"threads_started":1}
W20260807 18:33:50.460762 21216 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:33:50.454865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.67:39949 (local address 127.11.14.126:43679)
0807 18:33:50.455015 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:50.455021 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:50.455182 (+ 161us) server_negotiation.cc:407] Connection header received
0807 18:33:50.455278 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:50.455280 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:50.455333 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:50.455409 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:33:50.455895 (+ 486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:50.456456 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:50.457303 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:50.457526 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:50.457670 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:50.460652 (+ 2982us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.14.67:39949: BlockingRecv error: recv got EOF from 127.11.14.67:39949 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 07 18:33:51 dist-test-slave-6k22 krb5kdc[20129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.14.65@KRBTEST.COM for oryx/127.11.14.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:33:51 dist-test-slave-6k22 krb5kdc[20129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.14.65@KRBTEST.COM for oryx/127.11.14.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:33:51.061250 21219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:51.055712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:43679 (local address 127.11.14.65:35461)
0807 18:33:51.056028 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:51.056041 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:51.056141 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:51.056436 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:51.056439 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:51.056622 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:33:51.056724 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:51.056729 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:51.057534 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:51.057537 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:51.058085 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:51.058092 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:51.058519 (+ 427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:51.058522 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:51.058586 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:51.059151 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:33:51.059163 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:33:51.061076 (+ 1913us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.14.126:43679: Server oryx/127.11.14.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":123,"threads_started":1}
W20260807 18:33:51.061327 21220 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:33:51.055780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.65:35461 (local address 127.11.14.126:43679)
0807 18:33:51.056022 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:51.056025 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:51.056109 (+ 84us) server_negotiation.cc:407] Connection header received
0807 18:33:51.056242 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:51.056246 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:51.056309 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:51.056391 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:33:51.056836 (+ 445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:51.057427 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:51.058187 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:51.058418 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:51.058495 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:51.061201 (+ 2706us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.14.65:35461: BlockingRecv error: recv got EOF from 127.11.14.65:35461 (error 108)
Metrics: {"server-negotiator.queue_time_us":168,"thread_start_us":85,"threads_started":1}
Aug 07 18:33:51 dist-test-slave-6k22 krb5kdc[20129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.14.66@KRBTEST.COM for oryx/127.11.14.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:33:51 dist-test-slave-6k22 krb5kdc[20129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.14.66@KRBTEST.COM for oryx/127.11.14.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:33:51.263386 21221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:51.257356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:43679 (local address 127.11.14.66:43309)
0807 18:33:51.257665 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:51.257680 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:51.257822 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:51.258113 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:51.258116 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:51.258357 (+ 241us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:33:51.258458 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:51.258465 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:51.259242 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:51.259247 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:51.259846 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:51.259855 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:51.260319 (+ 464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:51.260322 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:51.260417 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:51.261001 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:33:51.261014 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:33:51.263196 (+ 2182us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.14.126:43679: Server oryx/127.11.14.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":96,"threads_started":1}
W20260807 18:33:51.263425 21220 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:33:51.257415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.66:43309 (local address 127.11.14.126:43679)
0807 18:33:51.257594 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:51.257599 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:51.257764 (+ 165us) server_negotiation.cc:407] Connection header received
0807 18:33:51.257937 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:51.257940 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:51.257994 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:51.258083 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:33:51.258586 (+ 503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:51.259125 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:51.259977 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:51.260202 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:51.260312 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:51.263329 (+ 3017us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.14.66:43309: BlockingRecv error: recv got EOF from 127.11.14.66:43309 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 07 18:33:51 dist-test-slave-6k22 krb5kdc[20129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.14.67@KRBTEST.COM for oryx/127.11.14.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:33:51 dist-test-slave-6k22 krb5kdc[20129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.14.67@KRBTEST.COM for oryx/127.11.14.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:33:51.467396 21222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:51.461610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:43679 (local address 127.11.14.67:60089)
0807 18:33:51.461998 (+ 388us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:51.462011 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:51.462142 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:51.462445 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:51.462448 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:51.462666 (+ 218us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:33:51.462764 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:51.462770 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:51.463615 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:51.463620 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:51.464192 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:51.464201 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:51.464685 (+ 484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:51.464689 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:51.464780 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:51.465376 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:33:51.465389 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:33:51.467258 (+ 1869us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.14.126:43679: Server oryx/127.11.14.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":317,"thread_start_us":88,"threads_started":1}
W20260807 18:33:51.467540 21220 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:33:51.461696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.67:60089 (local address 127.11.14.126:43679)
0807 18:33:51.461870 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:51.461875 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:51.462087 (+ 212us) server_negotiation.cc:407] Connection header received
0807 18:33:51.462250 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:51.462254 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:51.462312 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:51.462416 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:33:51.462894 (+ 478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:51.463435 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:51.464324 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:51.464562 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:51.464676 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:51.467411 (+ 2735us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.14.67:60089: BlockingRecv error: recv got EOF from 127.11.14.67:60089 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
Aug 07 18:33:52 dist-test-slave-6k22 krb5kdc[20129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.14.65@KRBTEST.COM for oryx/127.11.14.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:33:52 dist-test-slave-6k22 krb5kdc[20129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.14.65@KRBTEST.COM for oryx/127.11.14.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:33:52.068109 21223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:52.062219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:43679 (local address 127.11.14.65:56895)
0807 18:33:52.062537 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:52.062551 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:52.062641 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:52.062912 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:52.062914 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:52.063102 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:33:52.063199 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:52.063204 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:52.064003 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:52.064006 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:52.064673 (+ 667us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:52.064705 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:52.065320 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:52.065324 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:52.065436 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:52.066022 (+ 586us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:33:52.066034 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:33:52.067992 (+ 1958us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.14.126:43679: Server oryx/127.11.14.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":92,"threads_started":1}
W20260807 18:33:52.068291 21224 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:33:52.062343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.65:56895 (local address 127.11.14.126:43679)
0807 18:33:52.062591 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:52.062595 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:52.062609 (+ 14us) server_negotiation.cc:407] Connection header received
0807 18:33:52.062734 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:52.062737 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:52.062794 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:52.062873 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:33:52.063304 (+ 431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:52.063905 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:52.064828 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:52.065196 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:52.065313 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:52.068120 (+ 2807us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.14.65:56895: BlockingRecv error: recv got EOF from 127.11.14.65:56895 (error 108)
Metrics: {"server-negotiator.queue_time_us":181,"thread_start_us":59,"threads_started":1}
Aug 07 18:33:52 dist-test-slave-6k22 krb5kdc[20129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.14.66@KRBTEST.COM for oryx/127.11.14.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:33:52 dist-test-slave-6k22 krb5kdc[20129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.14.66@KRBTEST.COM for oryx/127.11.14.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:33:52.269423 21225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:52.264145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:43679 (local address 127.11.14.66:58671)
0807 18:33:52.264449 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:52.264463 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:52.264550 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:52.264801 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:52.264803 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:52.264999 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:33:52.265101 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:52.265107 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:52.265828 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:52.265831 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:52.266404 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:52.266412 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:52.266823 (+ 411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:52.266825 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:52.266884 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:52.267387 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:33:52.267398 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:33:52.269297 (+ 1899us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.14.126:43679: Server oryx/127.11.14.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":122,"threads_started":1}
W20260807 18:33:52.269588 21224 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:33:52.264345 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.66:58671 (local address 127.11.14.126:43679)
0807 18:33:52.264486 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:52.264490 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:52.264507 (+ 17us) server_negotiation.cc:407] Connection header received
0807 18:33:52.264639 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:52.264641 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:52.264686 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:52.264779 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:33:52.265205 (+ 426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:52.265728 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:52.266508 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:52.266723 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:52.266800 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:52.269463 (+ 2663us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.14.66:58671: BlockingRecv error: recv got EOF from 127.11.14.66:58671 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Aug 07 18:33:52 dist-test-slave-6k22 krb5kdc[20129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.14.67@KRBTEST.COM for oryx/127.11.14.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:33:52 dist-test-slave-6k22 krb5kdc[20129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.14.67@KRBTEST.COM for oryx/127.11.14.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:33:52.474601 21226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:52.468371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:43679 (local address 127.11.14.67:46185)
0807 18:33:52.468686 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:52.468700 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:52.468798 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:52.469042 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:52.469044 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:52.469323 (+ 279us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:33:52.469427 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:52.469433 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:52.470212 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:52.470234 (+ 22us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:52.470930 (+ 696us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:52.470942 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:52.471400 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:52.471403 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:52.471484 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:52.472168 (+ 684us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:33:52.472189 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:33:52.474460 (+ 2271us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.14.126:43679: Server oryx/127.11.14.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":132,"threads_started":1}
W20260807 18:33:52.474756 21224 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:33:52.468494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.67:46185 (local address 127.11.14.126:43679)
0807 18:33:52.468638 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:52.468642 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:52.468768 (+ 126us) server_negotiation.cc:407] Connection header received
0807 18:33:52.468881 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:52.468884 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:52.468932 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:52.469002 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:33:52.469533 (+ 531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:52.470078 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:52.471040 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:52.471255 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:52.471345 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:52.474600 (+ 3255us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.14.67:46185: BlockingRecv error: recv got EOF from 127.11.14.67:46185 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 07 18:33:53 dist-test-slave-6k22 krb5kdc[20129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.14.65@KRBTEST.COM for oryx/127.11.14.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:33:53 dist-test-slave-6k22 krb5kdc[20129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.14.65@KRBTEST.COM for oryx/127.11.14.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:33:53.074473 21227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:53.068854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:43679 (local address 127.11.14.65:58611)
0807 18:33:53.069169 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:53.069183 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:53.069315 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:53.069582 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:53.069584 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:53.069773 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:33:53.069876 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:53.069881 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:53.070716 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:53.070719 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:53.071289 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:53.071297 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:53.071808 (+ 511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:53.071811 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:53.071881 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:53.072411 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:33:53.072423 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:33:53.074344 (+ 1921us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.14.126:43679: Server oryx/127.11.14.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":73,"threads_started":1}
W20260807 18:33:53.074609 21228 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:33:53.068946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.65:58611 (local address 127.11.14.126:43679)
0807 18:33:53.069261 (+ 315us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:53.069264 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:53.069284 (+ 20us) server_negotiation.cc:407] Connection header received
0807 18:33:53.069415 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:53.069417 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:53.069463 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:53.069540 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:33:53.069987 (+ 447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:53.070609 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:53.071411 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:53.071703 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:53.071790 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:53.074478 (+ 2688us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.14.65:58611: BlockingRecv error: recv got EOF from 127.11.14.65:58611 (error 108)
Metrics: {"server-negotiator.queue_time_us":229,"thread_start_us":89,"threads_started":1}
Aug 07 18:33:53 dist-test-slave-6k22 krb5kdc[20129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.14.66@KRBTEST.COM for oryx/127.11.14.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:33:53 dist-test-slave-6k22 krb5kdc[20129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.14.66@KRBTEST.COM for oryx/127.11.14.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:33:53.276476 21229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:53.270213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:43679 (local address 127.11.14.66:39215)
0807 18:33:53.270560 (+ 347us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:53.270579 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:53.270701 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:53.271011 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:53.271014 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:53.271242 (+ 228us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:33:53.271341 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:53.271347 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:53.272223 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:53.272228 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:53.273037 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:53.273049 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:53.273685 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:53.273688 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:53.273752 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:53.274326 (+ 574us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:33:53.274347 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:33:53.276360 (+ 2013us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.14.126:43679: Server oryx/127.11.14.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":272,"thread_start_us":92,"threads_started":1}
W20260807 18:33:53.276670 21228 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:33:53.270378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.66:39215 (local address 127.11.14.126:43679)
0807 18:33:53.270559 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:53.270566 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:53.270640 (+ 74us) server_negotiation.cc:407] Connection header received
0807 18:33:53.270841 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:53.270844 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:53.270898 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:53.270987 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:33:53.271466 (+ 479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:53.272100 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:53.273181 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:53.273558 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:53.273669 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:53.276505 (+ 2836us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.14.66:39215: BlockingRecv error: recv got EOF from 127.11.14.66:39215 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 07 18:33:53 dist-test-slave-6k22 krb5kdc[20129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.14.67@KRBTEST.COM for oryx/127.11.14.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:33:53 dist-test-slave-6k22 krb5kdc[20129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.14.67@KRBTEST.COM for oryx/127.11.14.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:33:53.481259 21230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:53.475650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:43679 (local address 127.11.14.67:38929)
0807 18:33:53.475920 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:53.475934 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:53.476047 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:53.476387 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:53.476391 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:53.476623 (+ 232us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:33:53.476728 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:53.476734 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:53.477530 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:53.477532 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:53.478083 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:53.478090 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:53.478521 (+ 431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:53.478524 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:53.478590 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:53.479092 (+ 502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:33:53.479104 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:33:53.481077 (+ 1973us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.14.126:43679: Server oryx/127.11.14.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":99,"threads_started":1}
W20260807 18:33:53.481312 21228 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:33:53.475712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.67:38929 (local address 127.11.14.126:43679)
0807 18:33:53.475856 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:53.475860 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:53.476016 (+ 156us) server_negotiation.cc:407] Connection header received
0807 18:33:53.476151 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:53.476153 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:53.476203 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:53.476312 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:33:53.476842 (+ 530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:53.477430 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:53.478184 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:53.478415 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:53.478493 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:53.481206 (+ 2713us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.14.67:38929: BlockingRecv error: recv got EOF from 127.11.14.67:38929 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 07 18:33:54 dist-test-slave-6k22 krb5kdc[20129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.14.65@KRBTEST.COM for oryx/127.11.14.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:33:54 dist-test-slave-6k22 krb5kdc[20129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.14.65@KRBTEST.COM for oryx/127.11.14.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:33:54.080917 21231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:54.075280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:43679 (local address 127.11.14.65:45805)
0807 18:33:54.075733 (+ 453us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:54.075750 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:54.075875 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:54.076097 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:54.076099 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:54.076283 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:33:54.076381 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:54.076386 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:54.077187 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:54.077190 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:54.077857 (+ 667us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:54.077867 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:54.078280 (+ 413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:54.078284 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:54.078354 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:54.078851 (+ 497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:33:54.078863 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:33:54.080779 (+ 1916us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.14.126:43679: Server oryx/127.11.14.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":353,"thread_start_us":102,"threads_started":1}
W20260807 18:33:54.081120 21232 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:33:54.075413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.65:45805 (local address 127.11.14.126:43679)
0807 18:33:54.075754 (+ 341us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:54.075761 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:54.075826 (+ 65us) server_negotiation.cc:407] Connection header received
0807 18:33:54.075942 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:54.075944 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:54.075987 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:54.076063 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:33:54.076480 (+ 417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:54.077087 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:54.077966 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:54.078184 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:54.078260 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:54.080949 (+ 2689us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.14.65:45805: BlockingRecv error: recv got EOF from 127.11.14.65:45805 (error 108)
Metrics: {"server-negotiator.queue_time_us":237,"thread_start_us":94,"threads_started":1}
Aug 07 18:33:54 dist-test-slave-6k22 krb5kdc[20129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.14.66@KRBTEST.COM for oryx/127.11.14.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:33:54 dist-test-slave-6k22 krb5kdc[20129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.14.66@KRBTEST.COM for oryx/127.11.14.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:33:54.288054 21234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:54.277242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:43679 (local address 127.11.14.66:55933)
0807 18:33:54.277564 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:54.277577 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:54.277679 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:54.277945 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:54.277947 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:54.278129 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:33:54.278229 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:54.278234 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:54.279002 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:54.279005 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:54.279590 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:54.279598 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:54.280019 (+ 421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:54.280021 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:54.280082 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:54.286113 (+ 6031us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:33:54.286126 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:33:54.287934 (+ 1808us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.14.126:43679: Server oryx/127.11.14.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":102,"threads_started":1}
W20260807 18:33:54.288192 21232 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:33:54.277286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.66:55933 (local address 127.11.14.126:43679)
0807 18:33:54.277431 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:54.277435 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:54.277662 (+ 227us) server_negotiation.cc:407] Connection header received
0807 18:33:54.277754 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:54.277756 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:54.277836 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:54.277908 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:33:54.278330 (+ 422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:54.278902 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:54.279696 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:54.279922 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:54.279999 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:54.288071 (+ 8072us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.14.66:55933: BlockingRecv error: recv got EOF from 127.11.14.66:55933 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 07 18:33:54 dist-test-slave-6k22 krb5kdc[20129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.14.67@KRBTEST.COM for oryx/127.11.14.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:33:54 dist-test-slave-6k22 krb5kdc[20129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.14.67@KRBTEST.COM for oryx/127.11.14.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:33:54.487996 21235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:54.482199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:43679 (local address 127.11.14.67:36593)
0807 18:33:54.482535 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:54.482573 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:54.482718 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:54.483020 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:54.483023 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:54.483238 (+ 215us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:33:54.483346 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:54.483353 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:54.484201 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:54.484206 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:54.484785 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:54.484793 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:54.485279 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:54.485283 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:54.485388 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:54.485871 (+ 483us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:33:54.485884 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:33:54.487878 (+ 1994us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.14.126:43679: Server oryx/127.11.14.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":84,"threads_started":1}
W20260807 18:33:54.488147 21232 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:33:54.482511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.67:36593 (local address 127.11.14.126:43679)
0807 18:33:54.482657 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:54.482661 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:54.482676 (+ 15us) server_negotiation.cc:407] Connection header received
0807 18:33:54.482827 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:54.482843 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:54.482904 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:54.482989 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:33:54.483472 (+ 483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:54.484081 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:54.484913 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:54.485159 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:54.485273 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:54.488023 (+ 2750us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.14.67:36593: BlockingRecv error: recv got EOF from 127.11.14.67:36593 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Aug 07 18:33:55 dist-test-slave-6k22 krb5kdc[20129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.14.65@KRBTEST.COM for oryx/127.11.14.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:33:55 dist-test-slave-6k22 krb5kdc[20129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.14.65@KRBTEST.COM for oryx/127.11.14.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:33:55.087239 21236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:55.081623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:43679 (local address 127.11.14.65:55229)
0807 18:33:55.082071 (+ 448us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:55.082088 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:55.082249 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:55.082578 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:55.082583 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:55.082822 (+ 239us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:33:55.082961 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:55.082967 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:55.083813 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:55.083816 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:55.084369 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:55.084377 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:55.084807 (+ 430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:55.084809 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:55.084870 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:55.085341 (+ 471us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:33:55.085353 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:33:55.087125 (+ 1772us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.14.126:43679: Server oryx/127.11.14.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":358,"thread_start_us":155,"threads_started":1}
W20260807 18:33:55.087395 21237 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:33:55.081857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.65:55229 (local address 127.11.14.126:43679)
0807 18:33:55.082110 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:55.082114 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:55.082265 (+ 151us) server_negotiation.cc:407] Connection header received
0807 18:33:55.082375 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:55.082378 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:55.082437 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:55.082525 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:33:55.083072 (+ 547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:55.083699 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:55.084479 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:55.084697 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:55.084774 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:55.087257 (+ 2483us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.14.65:55229: BlockingRecv error: recv got EOF from 127.11.14.65:55229 (error 108)
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":72,"threads_started":1}
Aug 07 18:33:55 dist-test-slave-6k22 krb5kdc[20129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.14.66@KRBTEST.COM for oryx/127.11.14.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:33:55 dist-test-slave-6k22 krb5kdc[20129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.14.66@KRBTEST.COM for oryx/127.11.14.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:33:55.294847 21238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:55.288986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:43679 (local address 127.11.14.66:58843)
0807 18:33:55.289285 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:55.289298 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:55.289460 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:55.289736 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:55.289739 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:55.290005 (+ 266us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:33:55.290129 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:55.290136 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:55.291049 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:55.291054 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:55.291650 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:55.291659 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:55.292134 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:55.292138 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:55.292225 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:55.292746 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:33:55.292758 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:33:55.294673 (+ 1915us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.14.126:43679: Server oryx/127.11.14.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":87,"threads_started":1}
W20260807 18:33:55.294914 21237 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:33:55.289051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.66:58843 (local address 127.11.14.126:43679)
0807 18:33:55.289238 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:55.289243 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:55.289395 (+ 152us) server_negotiation.cc:407] Connection header received
0807 18:33:55.289566 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:55.289569 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:55.289624 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:55.289715 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:33:55.290256 (+ 541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:55.290930 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:55.291776 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:55.292015 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:55.292131 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:55.294793 (+ 2662us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.14.66:58843: BlockingRecv error: recv got EOF from 127.11.14.66:58843 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
Aug 07 18:33:55 dist-test-slave-6k22 krb5kdc[20129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.14.67@KRBTEST.COM for oryx/127.11.14.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:33:55 dist-test-slave-6k22 krb5kdc[20129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.14.67@KRBTEST.COM for oryx/127.11.14.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:33:55.494478 21239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:55.488846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:43679 (local address 127.11.14.67:56665)
0807 18:33:55.489152 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:55.489165 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:55.489299 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:55.489585 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:55.489588 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:55.489801 (+ 213us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:33:55.489904 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:55.489910 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:55.490682 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:55.490687 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:55.491266 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:55.491274 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:55.491831 (+ 557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:55.491835 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:55.491929 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:55.492456 (+ 527us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:33:55.492467 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:33:55.494275 (+ 1808us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.14.126:43679: Server oryx/127.11.14.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":112,"threads_started":1}
W20260807 18:33:55.494516 21237 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:33:55.488898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.67:56665 (local address 127.11.14.126:43679)
0807 18:33:55.489059 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:55.489064 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:55.489244 (+ 180us) server_negotiation.cc:407] Connection header received
0807 18:33:55.489417 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:55.489420 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:55.489474 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:55.489561 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:33:55.490028 (+ 467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:55.490559 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:55.491401 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:55.491692 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:55.491808 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:55.494407 (+ 2599us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.14.67:56665: BlockingRecv error: recv got EOF from 127.11.14.67:56665 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 07 18:33:56 dist-test-slave-6k22 krb5kdc[20129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.14.65@KRBTEST.COM for oryx/127.11.14.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:33:56 dist-test-slave-6k22 krb5kdc[20129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.14.65@KRBTEST.COM for oryx/127.11.14.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:33:56.093871 21240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:56.088062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:43679 (local address 127.11.14.65:59607)
0807 18:33:56.088356 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:56.088370 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:56.088466 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:56.088715 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:56.088717 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:56.088917 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:33:56.089018 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:56.089023 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:56.089992 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:56.089996 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:56.090551 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:56.090559 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:56.091050 (+ 491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:56.091053 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:56.091141 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:56.091750 (+ 609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:33:56.091762 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:33:56.093717 (+ 1955us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.14.126:43679: Server oryx/127.11.14.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":76,"threads_started":1}
W20260807 18:33:56.093998 21241 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:33:56.088110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.65:59607 (local address 127.11.14.126:43679)
0807 18:33:56.088393 (+ 283us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:56.088396 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:56.088412 (+ 16us) server_negotiation.cc:407] Connection header received
0807 18:33:56.088554 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:56.088556 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:56.088601 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:56.088680 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:33:56.089158 (+ 478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:56.089878 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:56.090691 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:56.090933 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:56.091046 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:56.093863 (+ 2817us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.14.65:59607: BlockingRecv error: recv got EOF from 127.11.14.65:59607 (error 108)
Metrics: {"server-negotiator.queue_time_us":202,"thread_start_us":97,"threads_started":1}
Aug 07 18:33:56 dist-test-slave-6k22 krb5kdc[20129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.14.66@KRBTEST.COM for oryx/127.11.14.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:33:56 dist-test-slave-6k22 krb5kdc[20129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.14.66@KRBTEST.COM for oryx/127.11.14.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:33:56.301743 21242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:56.295750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:43679 (local address 127.11.14.66:55533)
0807 18:33:56.296063 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:56.296076 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:56.296180 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:56.296437 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:56.296439 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:56.296627 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:33:56.296733 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:56.296738 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:56.297746 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:56.297753 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:56.298523 (+ 770us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:56.298534 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:56.298976 (+ 442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:56.298978 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:56.299037 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:56.299539 (+ 502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:33:56.299551 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:33:56.301623 (+ 2072us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.14.126:43679: Server oryx/127.11.14.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":109,"threads_started":1}
W20260807 18:33:56.301975 21241 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:33:56.295769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.66:55533 (local address 127.11.14.126:43679)
0807 18:33:56.295934 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:56.295937 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:56.296150 (+ 213us) server_negotiation.cc:407] Connection header received
0807 18:33:56.296275 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:56.296278 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:56.296323 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:56.296396 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:33:56.296876 (+ 480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:56.297618 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:56.298640 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:56.298872 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:56.298955 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:56.301776 (+ 2821us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.14.66:55533: BlockingRecv error: recv got EOF from 127.11.14.66:55533 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
Aug 07 18:33:56 dist-test-slave-6k22 krb5kdc[20129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.14.67@KRBTEST.COM for oryx/127.11.14.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:33:56 dist-test-slave-6k22 krb5kdc[20129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.14.67@KRBTEST.COM for oryx/127.11.14.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:33:56.500996 21243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:56.495320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:43679 (local address 127.11.14.67:45829)
0807 18:33:56.495695 (+ 375us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:56.495709 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:56.495811 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:56.496081 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:56.496083 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:56.496273 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:33:56.496393 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:56.496398 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:56.497239 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:56.497242 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:56.497804 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:56.497811 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:56.498317 (+ 506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:56.498319 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:56.498404 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:56.498975 (+ 571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:33:56.498988 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:33:56.500870 (+ 1882us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.14.126:43679: Server oryx/127.11.14.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":298,"thread_start_us":148,"threads_started":1}
W20260807 18:33:56.501183 21241 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:33:56.495373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.67:45829 (local address 127.11.14.126:43679)
0807 18:33:56.495580 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:56.495587 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:56.495794 (+ 207us) server_negotiation.cc:407] Connection header received
0807 18:33:56.495907 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:56.495909 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:56.495966 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:56.496039 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:33:56.496512 (+ 473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:56.497129 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:56.497924 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:56.498186 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:56.498283 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:56.501018 (+ 2735us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.14.67:45829: BlockingRecv error: recv got EOF from 127.11.14.67:45829 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
Aug 07 18:33:57 dist-test-slave-6k22 krb5kdc[20129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.14.65@KRBTEST.COM for oryx/127.11.14.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:33:57 dist-test-slave-6k22 krb5kdc[20129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.14.65@KRBTEST.COM for oryx/127.11.14.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:33:57.100646 21244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:57.094888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:43679 (local address 127.11.14.65:39775)
0807 18:33:57.095181 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:57.095196 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:57.095303 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:57.095633 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:57.095635 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:57.095888 (+ 253us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:33:57.095998 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:57.096004 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:57.096868 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:57.096873 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:57.097454 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:57.097462 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:57.097979 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:57.097983 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:57.098070 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:57.098596 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:33:57.098609 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:33:57.100525 (+ 1916us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.14.126:43679: Server oryx/127.11.14.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":92,"threads_started":1}
W20260807 18:33:57.100862 21245 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:33:57.094993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.65:39775 (local address 127.11.14.126:43679)
0807 18:33:57.095253 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:57.095257 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:57.095272 (+ 15us) server_negotiation.cc:407] Connection header received
0807 18:33:57.095405 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:57.095408 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:57.095453 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:57.095536 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:33:57.096140 (+ 604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:57.096732 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:57.097585 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:57.097849 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:57.097962 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:57.100670 (+ 2708us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.14.65:39775: BlockingRecv error: recv got EOF from 127.11.14.65:39775 (error 108)
Metrics: {"server-negotiator.queue_time_us":191,"thread_start_us":63,"threads_started":1}
Aug 07 18:33:57 dist-test-slave-6k22 krb5kdc[20129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.14.66@KRBTEST.COM for oryx/127.11.14.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:33:57 dist-test-slave-6k22 krb5kdc[20129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.14.66@KRBTEST.COM for oryx/127.11.14.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:33:57.308533 21246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:57.302622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:43679 (local address 127.11.14.66:41679)
0807 18:33:57.302984 (+ 362us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:57.303000 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:57.303125 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:57.303416 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:57.303419 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:57.303659 (+ 240us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:33:57.303760 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:57.303766 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:57.304623 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:57.304629 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:57.305310 (+ 681us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:57.305319 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:57.305786 (+ 467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:57.305789 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:57.305868 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:57.306449 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:33:57.306463 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:33:57.308356 (+ 1893us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.14.126:43679: Server oryx/127.11.14.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":281,"thread_start_us":107,"threads_started":1}
W20260807 18:33:57.308578 21245 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:33:57.302762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.66:41679 (local address 127.11.14.126:43679)
0807 18:33:57.302959 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:57.302965 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:57.303088 (+ 123us) server_negotiation.cc:407] Connection header received
0807 18:33:57.303216 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:57.303220 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:57.303274 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:57.303361 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:33:57.303899 (+ 538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:57.304509 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:57.305440 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:57.305667 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:57.305780 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:57.308477 (+ 2697us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.14.66:41679: BlockingRecv error: recv got EOF from 127.11.14.66:41679 (error 108)
Metrics: {"server-negotiator.queue_time_us":87}
Aug 07 18:33:57 dist-test-slave-6k22 krb5kdc[20129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.14.67@KRBTEST.COM for oryx/127.11.14.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:33:57 dist-test-slave-6k22 krb5kdc[20129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.14.67@KRBTEST.COM for oryx/127.11.14.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:33:57.507826 21248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:57.501937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:43679 (local address 127.11.14.67:42613)
0807 18:33:57.502389 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:57.502408 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:57.502526 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:57.502867 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:57.502873 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:57.503150 (+ 277us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:33:57.503283 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:57.503290 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:57.504071 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:57.504074 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:57.504631 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:57.504639 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:57.505059 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:57.505062 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:57.505124 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:57.505715 (+ 591us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:33:57.505736 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:33:57.507643 (+ 1907us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.14.126:43679: Server oryx/127.11.14.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":367,"thread_start_us":118,"threads_started":1}
W20260807 18:33:57.507865 21245 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:33:57.502065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.67:42613 (local address 127.11.14.126:43679)
0807 18:33:57.502202 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:57.502208 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:57.502478 (+ 270us) server_negotiation.cc:407] Connection header received
0807 18:33:57.502625 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:57.502627 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:57.502679 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:57.502791 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:33:57.503389 (+ 598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:57.503932 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:57.504738 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:57.504955 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:57.505031 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:57.507768 (+ 2737us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.14.67:42613: BlockingRecv error: recv got EOF from 127.11.14.67:42613 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Aug 07 18:33:58 dist-test-slave-6k22 krb5kdc[20129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.14.65@KRBTEST.COM for oryx/127.11.14.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:33:58 dist-test-slave-6k22 krb5kdc[20129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.14.65@KRBTEST.COM for oryx/127.11.14.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:33:58.106845 21250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:58.101505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:43679 (local address 127.11.14.65:56797)
0807 18:33:58.101845 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:58.101864 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:58.101980 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:58.102226 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:58.102227 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:58.102424 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:33:58.102524 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:58.102529 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:58.103253 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:58.103255 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:58.103854 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:58.103862 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:58.104270 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:58.104273 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:58.104346 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:58.104902 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:33:58.104917 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:33:58.106703 (+ 1786us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.14.126:43679: Server oryx/127.11.14.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":115,"threads_started":1}
W20260807 18:33:58.107023 21249 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:33:58.101564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.65:56797 (local address 127.11.14.126:43679)
0807 18:33:58.101848 (+ 284us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:58.101866 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:58.101935 (+ 69us) server_negotiation.cc:407] Connection header received
0807 18:33:58.102064 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:58.102066 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:58.102112 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:58.102189 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:33:58.102620 (+ 431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:58.103162 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:58.103953 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:58.104174 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:58.104250 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:58.106852 (+ 2602us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.14.65:56797: BlockingRecv error: recv got EOF from 127.11.14.65:56797 (error 108)
Metrics: {"server-negotiator.queue_time_us":184,"thread_start_us":94,"threads_started":1}
Aug 07 18:33:58 dist-test-slave-6k22 krb5kdc[20129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.14.66@KRBTEST.COM for oryx/127.11.14.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:33:58 dist-test-slave-6k22 krb5kdc[20129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.14.66@KRBTEST.COM for oryx/127.11.14.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:33:58.314699 21251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:58.309323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:43679 (local address 127.11.14.66:59325)
0807 18:33:58.309605 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:58.309617 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:58.309715 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:58.309935 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:58.309938 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:58.310123 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:33:58.310223 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:58.310228 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:58.310944 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:58.310947 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:58.311712 (+ 765us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:58.311724 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:58.312125 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:58.312128 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:58.312191 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:58.312686 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:33:58.312698 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:33:58.314550 (+ 1852us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.14.126:43679: Server oryx/127.11.14.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":108,"threads_started":1}
W20260807 18:33:58.314808 21249 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:33:58.309376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.66:59325 (local address 127.11.14.126:43679)
0807 18:33:58.309508 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:58.309512 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:58.309680 (+ 168us) server_negotiation.cc:407] Connection header received
0807 18:33:58.309792 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:58.309794 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:58.309842 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:58.309913 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:33:58.310316 (+ 403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:58.310849 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:58.311819 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:58.312032 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:58.312108 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:58.314677 (+ 2569us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.14.66:59325: BlockingRecv error: recv got EOF from 127.11.14.66:59325 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Aug 07 18:33:58 dist-test-slave-6k22 krb5kdc[20129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.14.67@KRBTEST.COM for oryx/127.11.14.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:33:58 dist-test-slave-6k22 krb5kdc[20129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.14.67@KRBTEST.COM for oryx/127.11.14.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:33:58.513772 21252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:58.508675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:43679 (local address 127.11.14.67:46557)
0807 18:33:58.508973 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:58.508987 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:58.509087 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:58.509316 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:58.509318 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:58.509502 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:33:58.509604 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:58.509609 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:58.510309 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:58.510311 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:58.510859 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:58.510866 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:58.511267 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:58.511270 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:58.511344 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:58.511879 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:33:58.511891 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:33:58.513654 (+ 1763us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.14.126:43679: Server oryx/127.11.14.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":122,"threads_started":1}
W20260807 18:33:58.513949 21249 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:33:58.508733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.67:46557 (local address 127.11.14.126:43679)
0807 18:33:58.508866 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:58.508870 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:58.509055 (+ 185us) server_negotiation.cc:407] Connection header received
0807 18:33:58.509153 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:58.509155 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:58.509203 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:58.509271 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:33:58.509700 (+ 429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:58.510207 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:58.510956 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:58.511173 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:58.511246 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:58.513793 (+ 2547us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.14.67:46557: BlockingRecv error: recv got EOF from 127.11.14.67:46557 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 07 18:33:59 dist-test-slave-6k22 krb5kdc[20129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.14.65@KRBTEST.COM for oryx/127.11.14.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:33:59 dist-test-slave-6k22 krb5kdc[20129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.14.65@KRBTEST.COM for oryx/127.11.14.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:33:59.112973 21253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:59.107666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:43679 (local address 127.11.14.65:34571)
0807 18:33:59.107969 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:59.107982 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:59.108073 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:59.108304 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:59.108306 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:59.108486 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:33:59.108586 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:59.108591 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:59.109353 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:59.109356 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:59.109911 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:59.109918 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:59.110331 (+ 413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:59.110335 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:59.110410 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:59.110957 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:33:59.110969 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:33:59.112855 (+ 1886us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.14.126:43679: Server oryx/127.11.14.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":112,"threads_started":1}
W20260807 18:33:59.113111 21254 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:33:59.107719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.65:34571 (local address 127.11.14.126:43679)
0807 18:33:59.108030 (+ 311us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:59.108034 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:59.108047 (+ 13us) server_negotiation.cc:407] Connection header received
0807 18:33:59.108157 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:59.108159 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:59.108202 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:59.108268 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:33:59.108687 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:59.109239 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:59.110009 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:59.110232 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:59.110322 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:59.112974 (+ 2652us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.14.65:34571: BlockingRecv error: recv got EOF from 127.11.14.65:34571 (error 108)
Metrics: {"server-negotiator.queue_time_us":244,"thread_start_us":84,"threads_started":1}
Aug 07 18:33:59 dist-test-slave-6k22 krb5kdc[20129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.14.66@KRBTEST.COM for oryx/127.11.14.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:33:59 dist-test-slave-6k22 krb5kdc[20129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.14.66@KRBTEST.COM for oryx/127.11.14.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:33:59.327091 21255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:59.315505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:43679 (local address 127.11.14.66:45201)
0807 18:33:59.315834 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:59.315848 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:59.315981 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:59.316253 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:59.316256 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:59.316477 (+ 221us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:33:59.316579 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:59.316584 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:59.317372 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:59.317378 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:59.317938 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:59.317947 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:59.318436 (+ 489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:59.318440 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:59.318527 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:59.324695 (+ 6168us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:33:59.324707 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:33:59.326967 (+ 2260us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.14.126:43679: Server oryx/127.11.14.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":135,"threads_started":1}
W20260807 18:33:59.327241 21254 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:33:59.315579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.66:45201 (local address 127.11.14.126:43679)
0807 18:33:59.315735 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:59.315740 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:59.315924 (+ 184us) server_negotiation.cc:407] Connection header received
0807 18:33:59.316081 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:59.316084 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:59.316136 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:59.316228 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:33:59.316700 (+ 472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:59.317241 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:59.318065 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:59.318312 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:59.318429 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:59.327119 (+ 8690us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.14.66:45201: BlockingRecv error: recv got EOF from 127.11.14.66:45201 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 07 18:33:59 dist-test-slave-6k22 krb5kdc[20129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.14.67@KRBTEST.COM for oryx/127.11.14.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:33:59 dist-test-slave-6k22 krb5kdc[20129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.14.67@KRBTEST.COM for oryx/127.11.14.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:33:59.519876 21257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:33:59.514626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:43679 (local address 127.11.14.67:41981)
0807 18:33:59.514942 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0807 18:33:59.514963 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 18:33:59.515062 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:33:59.515307 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:33:59.515309 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:33:59.515496 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:33:59.515618 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:59.515623 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:59.516335 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:59.516337 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:59.516888 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:33:59.516895 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:33:59.517300 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:33:59.517303 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:33:59.517366 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:59.517839 (+ 473us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:33:59.517850 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:33:59.519701 (+ 1851us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.14.126:43679: Server oryx/127.11.14.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":90,"threads_started":1}
W20260807 18:33:59.519941 21254 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:33:59.514722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.67:41981 (local address 127.11.14.126:43679)
0807 18:33:59.514882 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0807 18:33:59.514886 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:33:59.515028 (+ 142us) server_negotiation.cc:407] Connection header received
0807 18:33:59.515139 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:33:59.515141 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:33:59.515188 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:33:59.515267 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:33:59.515717 (+ 450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:59.516241 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:59.516990 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:33:59.517201 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:33:59.517278 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:33:59.519822 (+ 2544us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.14.67:41981: BlockingRecv error: recv got EOF from 127.11.14.67:41981 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Aug 07 18:34:00 dist-test-slave-6k22 krb5kdc[20129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.14.65@KRBTEST.COM for oryx/127.11.14.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:34:00 dist-test-slave-6k22 krb5kdc[20129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.14.65@KRBTEST.COM for oryx/127.11.14.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:34:00.120083 21258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:34:00.113805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:43679 (local address 127.11.14.65:42653)
0807 18:34:00.114098 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0807 18:34:00.114119 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 18:34:00.114210 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:34:00.114518 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:34:00.114520 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:34:00.114705 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:34:00.114807 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:34:00.114812 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:34:00.115977 (+ 1165us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:34:00.115980 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:34:00.116545 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:34:00.116553 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:34:00.117198 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:34:00.117201 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:34:00.117280 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:34:00.117853 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:34:00.117865 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:34:00.119964 (+ 2099us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.14.126:43679: Server oryx/127.11.14.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":110,"threads_started":1}
W20260807 18:34:00.120332 21259 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:34:00.113869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.65:42653 (local address 127.11.14.126:43679)
0807 18:34:00.114189 (+ 320us) server_negotiation.cc:206] Beginning negotiation
0807 18:34:00.114194 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:34:00.114213 (+ 19us) server_negotiation.cc:407] Connection header received
0807 18:34:00.114303 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:34:00.114308 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:34:00.114371 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:34:00.114480 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:34:00.114919 (+ 439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:34:00.115863 (+ 944us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:34:00.116660 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:34:00.117084 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:34:00.117205 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:34:00.120163 (+ 2958us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.14.65:42653: BlockingRecv error: recv got EOF from 127.11.14.65:42653 (error 108)
Metrics: {"server-negotiator.queue_time_us":232,"thread_start_us":117,"threads_started":1}
Aug 07 18:34:00 dist-test-slave-6k22 krb5kdc[20129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.14.66@KRBTEST.COM for oryx/127.11.14.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:34:00 dist-test-slave-6k22 krb5kdc[20129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.14.66@KRBTEST.COM for oryx/127.11.14.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:34:00.333773 21260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:34:00.328078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:43679 (local address 127.11.14.66:41689)
0807 18:34:00.328443 (+ 365us) negotiation.cc:107] Waiting for socket to connect
0807 18:34:00.328459 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 18:34:00.328605 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:34:00.328887 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:34:00.328890 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:34:00.329113 (+ 223us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:34:00.329207 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:34:00.329213 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:34:00.330098 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:34:00.330103 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:34:00.330671 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:34:00.330680 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:34:00.331139 (+ 459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:34:00.331143 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:34:00.331228 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:34:00.331810 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:34:00.331823 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:34:00.333653 (+ 1830us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.14.126:43679: Server oryx/127.11.14.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":286,"thread_start_us":132,"threads_started":1}
W20260807 18:34:00.334015 21259 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:34:00.328250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.66:41689 (local address 127.11.14.126:43679)
0807 18:34:00.328417 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0807 18:34:00.328423 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:34:00.328542 (+ 119us) server_negotiation.cc:407] Connection header received
0807 18:34:00.328723 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:34:00.328725 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:34:00.328771 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:34:00.328865 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:34:00.329334 (+ 469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:34:00.329985 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:34:00.330802 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:34:00.331024 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:34:00.331134 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:34:00.333806 (+ 2672us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.14.66:41689: BlockingRecv error: recv got EOF from 127.11.14.66:41689 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 07 18:34:00 dist-test-slave-6k22 krb5kdc[20129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.14.67@KRBTEST.COM for oryx/127.11.14.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:34:00 dist-test-slave-6k22 krb5kdc[20129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.14.67@KRBTEST.COM for oryx/127.11.14.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:34:00.526248 21261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:34:00.520666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:43679 (local address 127.11.14.67:60405)
0807 18:34:00.520992 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0807 18:34:00.521006 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 18:34:00.521114 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:34:00.521383 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:34:00.521386 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:34:00.521593 (+ 207us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:34:00.521743 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:34:00.521749 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:34:00.522486 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:34:00.522488 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:34:00.523079 (+ 591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:34:00.523087 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:34:00.523534 (+ 447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:34:00.523537 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:34:00.523704 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:34:00.524295 (+ 591us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:34:00.524308 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:34:00.526115 (+ 1807us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.14.126:43679: Server oryx/127.11.14.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":144,"threads_started":1}
W20260807 18:34:00.526412 21259 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:34:00.520747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.67:60405 (local address 127.11.14.126:43679)
0807 18:34:00.520914 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0807 18:34:00.520918 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:34:00.521080 (+ 162us) server_negotiation.cc:407] Connection header received
0807 18:34:00.521195 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:34:00.521197 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:34:00.521243 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:34:00.521331 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:34:00.521857 (+ 526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:34:00.522390 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:34:00.523180 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:34:00.523422 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:34:00.523498 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:34:00.526245 (+ 2747us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.14.67:60405: BlockingRecv error: recv got EOF from 127.11.14.67:60405 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Aug 07 18:34:01 dist-test-slave-6k22 krb5kdc[20129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.14.65@KRBTEST.COM for oryx/127.11.14.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:34:01 dist-test-slave-6k22 krb5kdc[20129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.14.65@KRBTEST.COM for oryx/127.11.14.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:34:01.126168 21262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:34:01.120815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:43679 (local address 127.11.14.65:42063)
0807 18:34:01.121108 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0807 18:34:01.121129 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 18:34:01.121247 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:34:01.121511 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:34:01.121513 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:34:01.121699 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:34:01.121804 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:34:01.121810 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:34:01.122558 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:34:01.122560 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:34:01.123129 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:34:01.123136 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:34:01.123605 (+ 469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:34:01.123612 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:34:01.123680 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:34:01.124190 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:34:01.124202 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:34:01.126040 (+ 1838us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.14.126:43679: Server oryx/127.11.14.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":75,"threads_started":1}
W20260807 18:34:01.126363 21263 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:34:01.120949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.65:42063 (local address 127.11.14.126:43679)
0807 18:34:01.121271 (+ 322us) server_negotiation.cc:206] Beginning negotiation
0807 18:34:01.121275 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:34:01.121293 (+ 18us) server_negotiation.cc:407] Connection header received
0807 18:34:01.121349 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:34:01.121351 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:34:01.121393 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:34:01.121471 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:34:01.121910 (+ 439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:34:01.122457 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:34:01.123230 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:34:01.123462 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:34:01.123538 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:34:01.126175 (+ 2637us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.14.65:42063: BlockingRecv error: recv got EOF from 127.11.14.65:42063 (error 108)
Metrics: {"server-negotiator.queue_time_us":250,"thread_start_us":81,"threads_started":1}
Aug 07 18:34:01 dist-test-slave-6k22 krb5kdc[20129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.14.66@KRBTEST.COM for oryx/127.11.14.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:34:01 dist-test-slave-6k22 krb5kdc[20129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.14.66@KRBTEST.COM for oryx/127.11.14.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:34:01.339674 21264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:34:01.334559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:43679 (local address 127.11.14.66:44807)
0807 18:34:01.334826 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0807 18:34:01.334841 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 18:34:01.334956 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:34:01.335164 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:34:01.335166 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:34:01.335355 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:34:01.335451 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:34:01.335456 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:34:01.336240 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:34:01.336243 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:34:01.336808 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:34:01.336816 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:34:01.337211 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:34:01.337214 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:34:01.337276 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:34:01.337755 (+ 479us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:34:01.337767 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:34:01.339498 (+ 1731us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.14.126:43679: Server oryx/127.11.14.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":96,"threads_started":1}
W20260807 18:34:01.339798 21263 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:34:01.334599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.66:44807 (local address 127.11.14.126:43679)
0807 18:34:01.334738 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0807 18:34:01.334742 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:34:01.334915 (+ 173us) server_negotiation.cc:407] Connection header received
0807 18:34:01.335034 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:34:01.335037 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:34:01.335081 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:34:01.335145 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:34:01.335544 (+ 399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:34:01.336129 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:34:01.336906 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:34:01.337116 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:34:01.337193 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:34:01.339683 (+ 2490us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.14.66:44807: BlockingRecv error: recv got EOF from 127.11.14.66:44807 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Aug 07 18:34:01 dist-test-slave-6k22 krb5kdc[20129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.14.67@KRBTEST.COM for oryx/127.11.14.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:34:01 dist-test-slave-6k22 krb5kdc[20129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.14.67@KRBTEST.COM for oryx/127.11.14.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:34:01.532125 21265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:34:01.527034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:43679 (local address 127.11.14.67:35617)
0807 18:34:01.527309 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0807 18:34:01.527332 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 18:34:01.527433 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:34:01.527644 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:34:01.527645 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:34:01.527839 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:34:01.527939 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:34:01.527945 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:34:01.528664 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:34:01.528666 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:34:01.529236 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:34:01.529244 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:34:01.529686 (+ 442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:34:01.529689 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:34:01.529754 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:34:01.530232 (+ 478us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:34:01.530243 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:34:01.531993 (+ 1750us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.14.126:43679: Server oryx/127.11.14.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":100,"threads_started":1}
W20260807 18:34:01.532281 21263 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:34:01.527067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.67:35617 (local address 127.11.14.126:43679)
0807 18:34:01.527206 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0807 18:34:01.527211 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:34:01.527389 (+ 178us) server_negotiation.cc:407] Connection header received
0807 18:34:01.527501 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:34:01.527503 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:34:01.527566 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:34:01.527631 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:34:01.528036 (+ 405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:34:01.528568 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:34:01.529348 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:34:01.529565 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:34:01.529641 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:34:01.532129 (+ 2488us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.14.67:35617: BlockingRecv error: recv got EOF from 127.11.14.67:35617 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 07 18:34:02 dist-test-slave-6k22 krb5kdc[20129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.14.65@KRBTEST.COM for oryx/127.11.14.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:34:02 dist-test-slave-6k22 krb5kdc[20129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.14.65@KRBTEST.COM for oryx/127.11.14.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:34:02.132707 21266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:34:02.127063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:43679 (local address 127.11.14.65:43905)
0807 18:34:02.127471 (+ 408us) negotiation.cc:107] Waiting for socket to connect
0807 18:34:02.127485 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 18:34:02.127607 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:34:02.127857 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:34:02.127859 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:34:02.128045 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:34:02.128153 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:34:02.128158 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:34:02.128912 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:34:02.128915 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:34:02.129464 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:34:02.129471 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:34:02.129879 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:34:02.129882 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:34:02.129946 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:34:02.130440 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:34:02.130462 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:34:02.132585 (+ 2123us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.14.126:43679: Server oryx/127.11.14.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":314,"thread_start_us":90,"threads_started":1}
W20260807 18:34:02.132941 21267 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:34:02.127125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.65:43905 (local address 127.11.14.126:43679)
0807 18:34:02.127479 (+ 354us) server_negotiation.cc:206] Beginning negotiation
0807 18:34:02.127483 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:34:02.127539 (+ 56us) server_negotiation.cc:407] Connection header received
0807 18:34:02.127678 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:34:02.127680 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:34:02.127722 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:34:02.127798 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:34:02.128261 (+ 463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:34:02.128813 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:34:02.129563 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:34:02.129782 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:34:02.129862 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:34:02.132779 (+ 2917us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.14.65:43905: BlockingRecv error: recv got EOF from 127.11.14.65:43905 (error 108)
Metrics: {"server-negotiator.queue_time_us":277,"thread_start_us":198,"threads_started":1}
Aug 07 18:34:02 dist-test-slave-6k22 krb5kdc[20129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.14.66@KRBTEST.COM for oryx/127.11.14.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:34:02 dist-test-slave-6k22 krb5kdc[20129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.14.66@KRBTEST.COM for oryx/127.11.14.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:34:02.347321 21268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:34:02.340669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:43679 (local address 127.11.14.66:60011)
0807 18:34:02.341088 (+ 419us) negotiation.cc:107] Waiting for socket to connect
0807 18:34:02.341126 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0807 18:34:02.341286 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:34:02.341626 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:34:02.341630 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:34:02.341876 (+ 246us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:34:02.341981 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:34:02.341988 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:34:02.342885 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:34:02.342892 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:34:02.343502 (+ 610us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:34:02.343511 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:34:02.344110 (+ 599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:34:02.344115 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:34:02.344212 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:34:02.344842 (+ 630us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:34:02.344857 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:34:02.347097 (+ 2240us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.14.126:43679: Server oryx/127.11.14.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":323,"thread_start_us":141,"threads_started":1}
W20260807 18:34:02.347358 21267 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:34:02.340829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.66:60011 (local address 127.11.14.126:43679)
0807 18:34:02.341041 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0807 18:34:02.341047 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:34:02.341220 (+ 173us) server_negotiation.cc:407] Connection header received
0807 18:34:02.341409 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:34:02.341414 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:34:02.341494 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:34:02.341607 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:34:02.342115 (+ 508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:34:02.342752 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:34:02.343728 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:34:02.343969 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:34:02.344099 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:34:02.347229 (+ 3130us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.14.66:60011: BlockingRecv error: recv got EOF from 127.11.14.66:60011 (error 108)
Metrics: {"server-negotiator.queue_time_us":87}
I20260807 18:34:02.425472 11321 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkz56XN/build/debug/bin/kudu with pid 20751
I20260807 18:34:02.438309 11321 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkz56XN/build/debug/bin/kudu with pid 20921
I20260807 18:34:02.446550 11321 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkz56XN/build/debug/bin/kudu with pid 21064
I20260807 18:34:02.453811 11321 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkz56XN/build/debug/bin/kudu with pid 20674
2026-08-07T18:34:02Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (18221 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-07T18:34:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:34:02Z Disabled control of system clock
I20260807 18:34:02.488583 11321 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkz56XN/build/debug/bin/kudu
/tmp/dist-test-taskkz56XN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786127579900523-11321-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786127579900523-11321-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786127579900523-11321-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786127579900523-11321-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.14.126:40347
--webserver_interface=127.11.14.126
--webserver_port=0
--builtin_ntp_servers=127.11.14.84:41789
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.14.126:40347
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260807 18:34:02.604012 21272 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:34:02.604293 21272 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:34:02.604331 21272 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:34:02.608837 21272 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:34:02.608901 21272 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:34:02.608917 21272 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:34:02.608935 21272 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:34:02.608952 21272 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:34:02.613921 21272 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.14.84:41789
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786127579900523-11321-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786127579900523-11321-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.14.126:40347
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.14.126:40347
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786127579900523-11321-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.14.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21272
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786127579900523-11321-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:34:02.614939 21272 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:34:02.616365 21272 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:34:02.625345 21281 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:34:02.625363 21278 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:34:02.625425 21272 server_base.cc:1034] running on GCE node
W20260807 18:34:02.625396 21277 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:34:02.626343 21272 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:34:02.627393 21272 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:34:02.628556 21272 hybrid_clock.cc:648] HybridClock initialized: now 1786127642628534 us; error 36 us; skew 500 ppm
I20260807 18:34:02.630474 21272 webserver.cc:460] Webserver started at http://127.11.14.126:44825/ using document root <none> and password file <none>
I20260807 18:34:02.631064 21272 fs_manager.cc:359] Metadata directory not provided
I20260807 18:34:02.631148 21272 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:34:02.631390 21272 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:34:02.633520 21272 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786127579900523-11321-0/minicluster-data/master-0/data/instance:
uuid: "82c7bebab3f741bf82219ab34bb81ba6"
format_stamp: "Formatted at 2026-08-07 18:34:02 on dist-test-slave-6k22"
server_key: "06c7408a5404ec151a0ce0f17cef701c"
server_key_iv: "5c7cc02241653472709ed4be7f2269f2"
server_key_version: "encryptionkey@0"
I20260807 18:34:02.634092 21272 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786127579900523-11321-0/minicluster-data/master-0/wal/instance:
uuid: "82c7bebab3f741bf82219ab34bb81ba6"
format_stamp: "Formatted at 2026-08-07 18:34:02 on dist-test-slave-6k22"
server_key: "06c7408a5404ec151a0ce0f17cef701c"
server_key_iv: "5c7cc02241653472709ed4be7f2269f2"
server_key_version: "encryptionkey@0"
I20260807 18:34:02.637864 21272 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.002s
I20260807 18:34:02.640311 21287 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:34:02.641527 21272 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 18:34:02.641652 21272 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786127579900523-11321-0/minicluster-data/master-0/data,/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786127579900523-11321-0/minicluster-data/master-0/wal
uuid: "82c7bebab3f741bf82219ab34bb81ba6"
format_stamp: "Formatted at 2026-08-07 18:34:02 on dist-test-slave-6k22"
server_key: "06c7408a5404ec151a0ce0f17cef701c"
server_key_iv: "5c7cc02241653472709ed4be7f2269f2"
server_key_version: "encryptionkey@0"
I20260807 18:34:02.641757 21272 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786127579900523-11321-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786127579900523-11321-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786127579900523-11321-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:34:02.660027 21272 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:34:02.660674 21272 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:34:02.660897 21272 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:34:02.667874 21272 rpc_server.cc:307] RPC server started. Bound to: 127.11.14.126:40347
I20260807 18:34:02.667896 21342 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.14.126:40347 every 8 connection(s)
I20260807 18:34:02.668813 21272 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786127579900523-11321-0/minicluster-data/master-0/data/info.pb
I20260807 18:34:02.672055 21343 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:34:02.674628 11321 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkz56XN/build/debug/bin/kudu as pid 21272
I20260807 18:34:02.674739 11321 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786127579900523-11321-0/minicluster-data/master-0/wal/instance
I20260807 18:34:02.675067 11321 external_mini_cluster.cc:1428] Setting key 2ced6aa07e2ec63f3026cadb56c55a36
I20260807 18:34:02.678223 21343 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 82c7bebab3f741bf82219ab34bb81ba6: Bootstrap starting.
I20260807 18:34:02.680531 21343 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 82c7bebab3f741bf82219ab34bb81ba6: Neither blocks nor log segments found. Creating new log.
I20260807 18:34:02.681252 21343 log.cc:824] T 00000000000000000000000000000000 P 82c7bebab3f741bf82219ab34bb81ba6: Log is configured to *not* fsync() on all Append() calls
I20260807 18:34:02.683024 21343 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 82c7bebab3f741bf82219ab34bb81ba6: No bootstrap required, opened a new log
I20260807 18:34:02.684571 21346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:34:02.676580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52020 (local address 127.11.14.126:40347)
0807 18:34:02.677221 (+ 641us) server_negotiation.cc:206] Beginning negotiation
0807 18:34:02.677236 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 18:34:02.677271 (+ 35us) server_negotiation.cc:407] Connection header received
0807 18:34:02.677886 (+ 615us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:34:02.677902 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:34:02.678395 (+ 493us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:34:02.678727 (+ 332us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:34:02.679166 (+ 439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:34:02.680254 (+ 1088us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:34:02.681184 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:34:02.681690 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:34:02.681874 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:34:02.682688 (+ 814us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:34:02.682693 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:34:02.682706 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:34:02.683082 (+ 376us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:34:02.683163 (+ 81us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:34:02.683191 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:34:02.683299 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0807 18:34:02.683462 (+ 163us) server_negotiation.cc:299] Negotiation successful
0807 18:34:02.683734 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":306,"thread_start_us":145,"threads_started":1}
I20260807 18:34:02.686425 21343 raft_consensus.cc:348] T 00000000000000000000000000000000 P 82c7bebab3f741bf82219ab34bb81ba6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82c7bebab3f741bf82219ab34bb81ba6" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 40347 } }
I20260807 18:34:02.686657 21343 raft_consensus.cc:374] T 00000000000000000000000000000000 P 82c7bebab3f741bf82219ab34bb81ba6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:34:02.686733 21343 raft_consensus.cc:729] T 00000000000000000000000000000000 P 82c7bebab3f741bf82219ab34bb81ba6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 82c7bebab3f741bf82219ab34bb81ba6, State: Initialized, Role: FOLLOWER
I20260807 18:34:02.687198 21343 consensus_queue.cc:260] T 00000000000000000000000000000000 P 82c7bebab3f741bf82219ab34bb81ba6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82c7bebab3f741bf82219ab34bb81ba6" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 40347 } }
I20260807 18:34:02.687381 21343 raft_consensus.cc:388] T 00000000000000000000000000000000 P 82c7bebab3f741bf82219ab34bb81ba6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:34:02.687495 21343 raft_consensus.cc:482] T 00000000000000000000000000000000 P 82c7bebab3f741bf82219ab34bb81ba6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:34:02.687636 21343 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 82c7bebab3f741bf82219ab34bb81ba6 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:34:02.688493 21343 raft_consensus.cc:504] T 00000000000000000000000000000000 P 82c7bebab3f741bf82219ab34bb81ba6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82c7bebab3f741bf82219ab34bb81ba6" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 40347 } }
I20260807 18:34:02.688829 21343 leader_election.cc:302] T 00000000000000000000000000000000 P 82c7bebab3f741bf82219ab34bb81ba6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 82c7bebab3f741bf82219ab34bb81ba6; no voters:
I20260807 18:34:02.689118 21343 leader_election.cc:288] T 00000000000000000000000000000000 P 82c7bebab3f741bf82219ab34bb81ba6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:34:02.689255 21348 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 82c7bebab3f741bf82219ab34bb81ba6 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:34:02.689510 21348 raft_consensus.cc:686] T 00000000000000000000000000000000 P 82c7bebab3f741bf82219ab34bb81ba6 [term 1 LEADER]: Becoming Leader. State: Replica: 82c7bebab3f741bf82219ab34bb81ba6, State: Running, Role: LEADER
I20260807 18:34:02.689870 21348 consensus_queue.cc:237] T 00000000000000000000000000000000 P 82c7bebab3f741bf82219ab34bb81ba6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82c7bebab3f741bf82219ab34bb81ba6" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 40347 } }
I20260807 18:34:02.690399 21343 sys_catalog.cc:564] T 00000000000000000000000000000000 P 82c7bebab3f741bf82219ab34bb81ba6 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:34:02.691509 21349 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82c7bebab3f741bf82219ab34bb81ba6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "82c7bebab3f741bf82219ab34bb81ba6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82c7bebab3f741bf82219ab34bb81ba6" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 40347 } } }
I20260807 18:34:02.691653 21349 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82c7bebab3f741bf82219ab34bb81ba6 [sys.catalog]: This master's current role is: LEADER
I20260807 18:34:02.692164 21350 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82c7bebab3f741bf82219ab34bb81ba6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 82c7bebab3f741bf82219ab34bb81ba6. Latest consensus state: current_term: 1 leader_uuid: "82c7bebab3f741bf82219ab34bb81ba6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82c7bebab3f741bf82219ab34bb81ba6" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 40347 } } }
I20260807 18:34:02.692269 21350 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82c7bebab3f741bf82219ab34bb81ba6 [sys.catalog]: This master's current role is: LEADER
I20260807 18:34:02.692891 21357 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:34:02.695443 21357 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:34:02.700855 21357 catalog_manager.cc:1347] Generated new cluster ID: 544820e2fc02438387216669b01acca6
I20260807 18:34:02.700920 21357 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:34:02.733721 21357 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:34:02.734695 21357 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:34:02.742736 21357 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 82c7bebab3f741bf82219ab34bb81ba6: Generated new TSK 0
I20260807 18:34:02.743494 21357 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:34:02.750458 11321 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkz56XN/build/debug/bin/kudu
/tmp/dist-test-taskkz56XN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786127579900523-11321-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786127579900523-11321-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786127579900523-11321-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786127579900523-11321-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.14.65:0
--local_ip_for_outbound_sockets=127.11.14.65
--webserver_interface=127.11.14.65
--webserver_port=0
--tserver_master_addrs=127.11.14.126:40347
--builtin_ntp_servers=127.11.14.84:41789
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260807 18:34:02.871706 21367 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:34:02.871994 21367 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:34:02.872064 21367 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:34:02.876106 21367 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:34:02.876174 21367 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:34:02.876256 21367 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.14.65
I20260807 18:34:02.881115 21367 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.14.84:41789
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786127579900523-11321-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786127579900523-11321-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.14.65:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786127579900523-11321-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.14.65
--webserver_port=0
--tserver_master_addrs=127.11.14.126:40347
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21367
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.14.65
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786127579900523-11321-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:34:02.882166 21367 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:34:02.883539 21367 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:34:02.893272 21373 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:34:02.893314 21374 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:34:02.893450 21367 server_base.cc:1034] running on GCE node
W20260807 18:34:02.893648 21376 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:34:02.894093 21367 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:34:02.894899 21367 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:34:02.896066 21367 hybrid_clock.cc:648] HybridClock initialized: now 1786127642896030 us; error 51 us; skew 500 ppm
I20260807 18:34:02.898025 21367 webserver.cc:460] Webserver started at http://127.11.14.65:33089/ using document root <none> and password file <none>
I20260807 18:34:02.898635 21367 fs_manager.cc:359] Metadata directory not provided
I20260807 18:34:02.898699 21367 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:34:02.898939 21367 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:34:02.901058 21367 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786127579900523-11321-0/minicluster-data/ts-0/data/instance:
uuid: "491bb733d15c4850bbcb788e751d9c97"
format_stamp: "Formatted at 2026-08-07 18:34:02 on dist-test-slave-6k22"
server_key: "80a1bf8d57bef2b3f7d4b67a60b7d041"
server_key_iv: "40fee37363ab97da0049c6a9f136651c"
server_key_version: "encryptionkey@0"
I20260807 18:34:02.901657 21367 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786127579900523-11321-0/minicluster-data/ts-0/wal/instance:
uuid: "491bb733d15c4850bbcb788e751d9c97"
format_stamp: "Formatted at 2026-08-07 18:34:02 on dist-test-slave-6k22"
server_key: "80a1bf8d57bef2b3f7d4b67a60b7d041"
server_key_iv: "40fee37363ab97da0049c6a9f136651c"
server_key_version: "encryptionkey@0"
I20260807 18:34:02.905938 21367 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 18:34:02.908692 21382 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:34:02.909940 21367 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 18:34:02.910076 21367 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786127579900523-11321-0/minicluster-data/ts-0/data,/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786127579900523-11321-0/minicluster-data/ts-0/wal
uuid: "491bb733d15c4850bbcb788e751d9c97"
format_stamp: "Formatted at 2026-08-07 18:34:02 on dist-test-slave-6k22"
server_key: "80a1bf8d57bef2b3f7d4b67a60b7d041"
server_key_iv: "40fee37363ab97da0049c6a9f136651c"
server_key_version: "encryptionkey@0"
I20260807 18:34:02.910192 21367 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786127579900523-11321-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786127579900523-11321-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786127579900523-11321-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:34:02.931361 21367 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:34:02.932135 21367 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:34:02.932346 21367 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:34:02.932953 21367 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:34:02.933898 21367 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:34:02.933969 21367 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:34:02.934037 21367 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:34:02.934084 21367 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:34:02.943146 21367 rpc_server.cc:307] RPC server started. Bound to: 127.11.14.65:34623
I20260807 18:34:02.943173 21503 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.14.65:34623 every 8 connection(s)
I20260807 18:34:02.944103 21367 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786127579900523-11321-0/minicluster-data/ts-0/data/info.pb
I20260807 18:34:02.948153 11321 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkz56XN/build/debug/bin/kudu as pid 21367
I20260807 18:34:02.948247 11321 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786127579900523-11321-0/minicluster-data/ts-0/wal/instance
I20260807 18:34:02.948683 11321 external_mini_cluster.cc:1428] Setting key aa8b95a77d94d899ddfe9c504a9dfa6b
I20260807 18:34:02.950889 11321 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkz56XN/build/debug/bin/kudu
/tmp/dist-test-taskkz56XN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786127579900523-11321-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786127579900523-11321-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786127579900523-11321-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786127579900523-11321-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.14.66:0
--local_ip_for_outbound_sockets=127.11.14.66
--webserver_interface=127.11.14.66
--webserver_port=0
--tserver_master_addrs=127.11.14.126:40347
--builtin_ntp_servers=127.11.14.84:41789
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260807 18:34:02.954339 21346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:34:02.946123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.65:38987 (local address 127.11.14.126:40347)
0807 18:34:02.946287 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0807 18:34:02.946293 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:34:02.947447 (+ 1154us) server_negotiation.cc:407] Connection header received
0807 18:34:02.949042 (+ 1595us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:34:02.949048 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:34:02.949111 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:34:02.949191 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:34:02.950399 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:34:02.951179 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:34:02.952399 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:34:02.952633 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:34:02.952745 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:34:02.953847 (+ 1102us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:34:02.953850 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:34:02.953853 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:34:02.953866 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:34:02.953894 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:34:02.953903 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:34:02.953976 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0807 18:34:02.954136 (+ 160us) server_negotiation.cc:299] Negotiation successful
0807 18:34:02.954200 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260807 18:34:02.955188 21506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:34:02.946595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:40347 (local address 127.11.14.65:38987)
0807 18:34:02.947266 (+ 671us) negotiation.cc:107] Waiting for socket to connect
0807 18:34:02.947325 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0807 18:34:02.948817 (+ 1492us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:34:02.949404 (+ 587us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:34:02.949411 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:34:02.949524 (+ 113us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:34:02.950217 (+ 693us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:34:02.950241 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:34:02.951315 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:34:02.951318 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:34:02.952256 (+ 938us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:34:02.952266 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:34:02.952787 (+ 521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:34:02.952791 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:34:02.952958 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:34:02.953585 (+ 627us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 18:34:02.953586 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:34:02.953627 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 18:34:02.953693 (+ 66us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:34:02.953990 (+ 297us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:34:02.954001 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:34:02.954012 (+ 11us) client_negotiation.cc:769] Sending connection context
0807 18:34:02.954134 (+ 122us) client_negotiation.cc:240] Negotiation successful
0807 18:34:02.954431 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":117,"threads_started":1}
I20260807 18:34:02.956681 21504 heartbeater.cc:347] Connected to a master server at 127.11.14.126:40347
I20260807 18:34:02.956928 21504 heartbeater.cc:464] Registering TS with master...
I20260807 18:34:02.957449 21504 heartbeater.cc:511] Master 127.11.14.126:40347 requested a full tablet report, sending...
I20260807 18:34:02.958954 21305 ts_manager.cc:194] Registered new tserver with Master: 491bb733d15c4850bbcb788e751d9c97 (127.11.14.65:34623)
I20260807 18:34:02.960121 21305 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.14.65:38987
W20260807 18:34:03.066668 21507 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:34:03.066947 21507 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:34:03.067016 21507 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:34:03.071020 21507 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:34:03.071094 21507 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:34:03.071180 21507 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.14.66
I20260807 18:34:03.075950 21507 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.14.84:41789
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786127579900523-11321-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786127579900523-11321-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.14.66:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786127579900523-11321-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.14.66
--webserver_port=0
--tserver_master_addrs=127.11.14.126:40347
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21507
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.14.66
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786127579900523-11321-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:34:03.076980 21507 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:34:03.078457 21507 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:34:03.088466 21513 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:34:03.088474 21515 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:34:03.088474 21512 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:34:03.088752 21507 server_base.cc:1034] running on GCE node
I20260807 18:34:03.089229 21507 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:34:03.089887 21507 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:34:03.091075 21507 hybrid_clock.cc:648] HybridClock initialized: now 1786127643091034 us; error 51 us; skew 500 ppm
I20260807 18:34:03.093106 21507 webserver.cc:460] Webserver started at http://127.11.14.66:38069/ using document root <none> and password file <none>
I20260807 18:34:03.093694 21507 fs_manager.cc:359] Metadata directory not provided
I20260807 18:34:03.093786 21507 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:34:03.094018 21507 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:34:03.096128 21507 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786127579900523-11321-0/minicluster-data/ts-1/data/instance:
uuid: "6d466d01720347cdab5158ee9ee74169"
format_stamp: "Formatted at 2026-08-07 18:34:03 on dist-test-slave-6k22"
server_key: "26bcfaec2c73b2ba3a8b991301576584"
server_key_iv: "b52c6e3c0352b4670c86c4fa2d6a308a"
server_key_version: "encryptionkey@0"
I20260807 18:34:03.096710 21507 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786127579900523-11321-0/minicluster-data/ts-1/wal/instance:
uuid: "6d466d01720347cdab5158ee9ee74169"
format_stamp: "Formatted at 2026-08-07 18:34:03 on dist-test-slave-6k22"
server_key: "26bcfaec2c73b2ba3a8b991301576584"
server_key_iv: "b52c6e3c0352b4670c86c4fa2d6a308a"
server_key_version: "encryptionkey@0"
I20260807 18:34:03.100548 21507 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 18:34:03.103085 21521 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:34:03.104468 21507 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260807 18:34:03.104601 21507 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786127579900523-11321-0/minicluster-data/ts-1/data,/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786127579900523-11321-0/minicluster-data/ts-1/wal
uuid: "6d466d01720347cdab5158ee9ee74169"
format_stamp: "Formatted at 2026-08-07 18:34:03 on dist-test-slave-6k22"
server_key: "26bcfaec2c73b2ba3a8b991301576584"
server_key_iv: "b52c6e3c0352b4670c86c4fa2d6a308a"
server_key_version: "encryptionkey@0"
I20260807 18:34:03.104715 21507 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786127579900523-11321-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786127579900523-11321-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786127579900523-11321-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:34:03.130759 21507 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:34:03.131515 21507 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:34:03.131762 21507 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:34:03.132366 21507 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:34:03.133340 21507 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:34:03.133411 21507 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:34:03.133482 21507 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:34:03.133523 21507 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:34:03.142445 21507 rpc_server.cc:307] RPC server started. Bound to: 127.11.14.66:39251
I20260807 18:34:03.142515 21637 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.14.66:39251 every 8 connection(s)
I20260807 18:34:03.143364 21507 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786127579900523-11321-0/minicluster-data/ts-1/data/info.pb
I20260807 18:34:03.148448 11321 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkz56XN/build/debug/bin/kudu as pid 21507
I20260807 18:34:03.148571 11321 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786127579900523-11321-0/minicluster-data/ts-1/wal/instance
I20260807 18:34:03.148867 11321 external_mini_cluster.cc:1428] Setting key 0c96d0c60659989010a1b3392b7d4fae
I20260807 18:34:03.150890 11321 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkz56XN/build/debug/bin/kudu
/tmp/dist-test-taskkz56XN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786127579900523-11321-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786127579900523-11321-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786127579900523-11321-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786127579900523-11321-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.14.67:0
--local_ip_for_outbound_sockets=127.11.14.67
--webserver_interface=127.11.14.67
--webserver_port=0
--tserver_master_addrs=127.11.14.126:40347
--builtin_ntp_servers=127.11.14.84:41789
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260807 18:34:03.153142 21346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:34:03.145277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.66:35481 (local address 127.11.14.126:40347)
0807 18:34:03.145409 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0807 18:34:03.145413 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:34:03.146558 (+ 1145us) server_negotiation.cc:407] Connection header received
0807 18:34:03.147983 (+ 1425us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:34:03.147986 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:34:03.148035 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:34:03.148106 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:34:03.149116 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:34:03.149790 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:34:03.151146 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:34:03.151380 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:34:03.151495 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:34:03.152652 (+ 1157us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:34:03.152659 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:34:03.152665 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:34:03.152684 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:34:03.152713 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:34:03.152733 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:34:03.152805 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0807 18:34:03.152987 (+ 182us) server_negotiation.cc:299] Negotiation successful
0807 18:34:03.153036 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260807 18:34:03.154346 21640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:34:03.145677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:40347 (local address 127.11.14.66:35481)
0807 18:34:03.146423 (+ 746us) negotiation.cc:107] Waiting for socket to connect
0807 18:34:03.146464 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 18:34:03.147765 (+ 1301us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:34:03.148302 (+ 537us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:34:03.148310 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:34:03.148391 (+ 81us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:34:03.148959 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:34:03.148980 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:34:03.149922 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:34:03.149926 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:34:03.150879 (+ 953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:34:03.150890 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:34:03.151538 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:34:03.151542 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:34:03.151731 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:34:03.152361 (+ 630us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 18:34:03.152362 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:34:03.152401 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 18:34:03.152462 (+ 61us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:34:03.152836 (+ 374us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:34:03.152845 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:34:03.152852 (+ 7us) client_negotiation.cc:769] Sending connection context
0807 18:34:03.152983 (+ 131us) client_negotiation.cc:240] Negotiation successful
0807 18:34:03.153294 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":116,"threads_started":1}
I20260807 18:34:03.155483 21638 heartbeater.cc:347] Connected to a master server at 127.11.14.126:40347
I20260807 18:34:03.155743 21638 heartbeater.cc:464] Registering TS with master...
I20260807 18:34:03.156297 21638 heartbeater.cc:511] Master 127.11.14.126:40347 requested a full tablet report, sending...
I20260807 18:34:03.157375 21305 ts_manager.cc:194] Registered new tserver with Master: 6d466d01720347cdab5158ee9ee74169 (127.11.14.66:39251)
I20260807 18:34:03.157933 21305 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.14.66:35481
W20260807 18:34:03.267191 21641 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:34:03.267477 21641 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:34:03.267545 21641 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:34:03.271610 21641 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:34:03.271692 21641 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:34:03.271790 21641 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.14.67
I20260807 18:34:03.276435 21641 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.14.84:41789
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786127579900523-11321-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786127579900523-11321-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.14.67:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786127579900523-11321-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.14.67
--webserver_port=0
--tserver_master_addrs=127.11.14.126:40347
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21641
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.14.67
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786127579900523-11321-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:34:03.277498 21641 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:34:03.278945 21641 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:34:03.288317 21649 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:34:03.288321 21647 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:34:03.288342 21646 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:34:03.288694 21641 server_base.cc:1034] running on GCE node
I20260807 18:34:03.289125 21641 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:34:03.289824 21641 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:34:03.291015 21641 hybrid_clock.cc:648] HybridClock initialized: now 1786127643290993 us; error 26 us; skew 500 ppm
I20260807 18:34:03.292968 21641 webserver.cc:460] Webserver started at http://127.11.14.67:40083/ using document root <none> and password file <none>
I20260807 18:34:03.293555 21641 fs_manager.cc:359] Metadata directory not provided
I20260807 18:34:03.293619 21641 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:34:03.293830 21641 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:34:03.295917 21641 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786127579900523-11321-0/minicluster-data/ts-2/data/instance:
uuid: "dc0bf92bca9d43118b33cc6a25347ed4"
format_stamp: "Formatted at 2026-08-07 18:34:03 on dist-test-slave-6k22"
server_key: "b7188da4d419ba6315d51bd74b198e68"
server_key_iv: "56a85b0dadee42197ead686b5a71f6e7"
server_key_version: "encryptionkey@0"
I20260807 18:34:03.296448 21641 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786127579900523-11321-0/minicluster-data/ts-2/wal/instance:
uuid: "dc0bf92bca9d43118b33cc6a25347ed4"
format_stamp: "Formatted at 2026-08-07 18:34:03 on dist-test-slave-6k22"
server_key: "b7188da4d419ba6315d51bd74b198e68"
server_key_iv: "56a85b0dadee42197ead686b5a71f6e7"
server_key_version: "encryptionkey@0"
I20260807 18:34:03.300308 21641 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 18:34:03.302925 21656 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:34:03.304308 21641 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 18:34:03.304440 21641 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786127579900523-11321-0/minicluster-data/ts-2/data,/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786127579900523-11321-0/minicluster-data/ts-2/wal
uuid: "dc0bf92bca9d43118b33cc6a25347ed4"
format_stamp: "Formatted at 2026-08-07 18:34:03 on dist-test-slave-6k22"
server_key: "b7188da4d419ba6315d51bd74b198e68"
server_key_iv: "56a85b0dadee42197ead686b5a71f6e7"
server_key_version: "encryptionkey@0"
I20260807 18:34:03.304553 21641 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786127579900523-11321-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786127579900523-11321-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786127579900523-11321-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:34:03.337014 21641 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:34:03.337795 21641 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:34:03.338001 21641 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:34:03.338670 21641 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:34:03.339702 21641 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:34:03.339756 21641 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:34:03.339843 21641 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:34:03.339879 21641 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:34:03.349048 21641 rpc_server.cc:307] RPC server started. Bound to: 127.11.14.67:36969
I20260807 18:34:03.349082 21777 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.14.67:36969 every 8 connection(s)
I20260807 18:34:03.349975 21641 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786127579900523-11321-0/minicluster-data/ts-2/data/info.pb
I20260807 18:34:03.359211 11321 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkz56XN/build/debug/bin/kudu as pid 21641
I20260807 18:34:03.359368 11321 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786127579900523-11321-0/minicluster-data/ts-2/wal/instance
I20260807 18:34:03.359511 21346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:34:03.351807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.67:47693 (local address 127.11.14.126:40347)
0807 18:34:03.351953 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0807 18:34:03.351957 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:34:03.353024 (+ 1067us) server_negotiation.cc:407] Connection header received
0807 18:34:03.354494 (+ 1470us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:34:03.354497 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:34:03.354563 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:34:03.354666 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:34:03.355684 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:34:03.356225 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:34:03.357456 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:34:03.357686 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:34:03.357795 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:34:03.358985 (+ 1190us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:34:03.358989 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:34:03.358992 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:34:03.359006 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:34:03.359033 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:34:03.359041 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:34:03.359117 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0807 18:34:03.359289 (+ 172us) server_negotiation.cc:299] Negotiation successful
0807 18:34:03.359357 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260807 18:34:03.359736 11321 external_mini_cluster.cc:1428] Setting key 9d32a78efe3390493fff31fd6133a442
I20260807 18:34:03.360661 21780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:34:03.352187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:40347 (local address 127.11.14.67:47693)
0807 18:34:03.352861 (+ 674us) negotiation.cc:107] Waiting for socket to connect
0807 18:34:03.352905 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 18:34:03.354247 (+ 1342us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:34:03.354863 (+ 616us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:34:03.354871 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:34:03.354965 (+ 94us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:34:03.355482 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:34:03.355495 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:34:03.356373 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:34:03.356376 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:34:03.357314 (+ 938us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:34:03.357325 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:34:03.357858 (+ 533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:34:03.357862 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:34:03.358038 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:34:03.358702 (+ 664us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 18:34:03.358704 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:34:03.358747 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 18:34:03.358817 (+ 70us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:34:03.359133 (+ 316us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:34:03.359145 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:34:03.359156 (+ 11us) client_negotiation.cc:769] Sending connection context
0807 18:34:03.359286 (+ 130us) client_negotiation.cc:240] Negotiation successful
0807 18:34:03.359633 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":113,"threads_started":1}
I20260807 18:34:03.361821 21778 heartbeater.cc:347] Connected to a master server at 127.11.14.126:40347
I20260807 18:34:03.362071 21778 heartbeater.cc:464] Registering TS with master...
I20260807 18:34:03.362609 21778 heartbeater.cc:511] Master 127.11.14.126:40347 requested a full tablet report, sending...
I20260807 18:34:03.363725 21305 ts_manager.cc:194] Registered new tserver with Master: dc0bf92bca9d43118b33cc6a25347ed4 (127.11.14.67:36969)
I20260807 18:34:03.364298 21305 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.14.67:47693
I20260807 18:34:03.373831 11321 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260807 18:34:03.376668 21346 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:34:03.375842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52034 (local address 127.11.14.126:40347)
0807 18:34:03.375975 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0807 18:34:03.375979 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:34:03.376101 (+ 122us) server_negotiation.cc:407] Connection header received
0807 18:34:03.376214 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:34:03.376216 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:34:03.376256 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:34:03.376331 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:34:03.376553 (+ 222us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52034: BlockingRecv error: recv got EOF from 127.0.0.1:52034 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
I20260807 18:34:03.376817 11321 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkz56XN/build/debug/bin/kudu with pid 21367
I20260807 18:34:03.386917 11321 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkz56XN/build/debug/bin/kudu with pid 21507
I20260807 18:34:03.396011 11321 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkz56XN/build/debug/bin/kudu with pid 21641
I20260807 18:34:03.404515 11321 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkz56XN/build/debug/bin/kudu with pid 21272
2026-08-07T18:34:03Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (950 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-07T18:34:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:34:03Z Disabled control of system clock
I20260807 18:34:03.438598 11321 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkz56XN/build/debug/bin/kudu
/tmp/dist-test-taskkz56XN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786127579900523-11321-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786127579900523-11321-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786127579900523-11321-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786127579900523-11321-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.14.126:43697
--webserver_interface=127.11.14.126
--webserver_port=0
--builtin_ntp_servers=127.11.14.84:43109
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.14.126:43697
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260807 18:34:03.554716 21789 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:34:03.554965 21789 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:34:03.555001 21789 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:34:03.558876 21789 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:34:03.558926 21789 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:34:03.558941 21789 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:34:03.558957 21789 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:34:03.558972 21789 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:34:03.563604 21789 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.14.84:43109
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786127579900523-11321-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786127579900523-11321-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.14.126:43697
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.14.126:43697
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786127579900523-11321-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.14.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21789
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786127579900523-11321-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:34:03.564603 21789 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:34:03.565882 21789 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:34:03.574265 21795 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:34:03.574352 21797 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:34:03.574302 21794 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:34:03.574615 21789 server_base.cc:1034] running on GCE node
I20260807 18:34:03.575201 21789 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:34:03.576342 21789 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:34:03.577538 21789 hybrid_clock.cc:648] HybridClock initialized: now 1786127643577507 us; error 45 us; skew 500 ppm
I20260807 18:34:03.579351 21789 webserver.cc:460] Webserver started at http://127.11.14.126:42207/ using document root <none> and password file <none>
I20260807 18:34:03.579978 21789 fs_manager.cc:359] Metadata directory not provided
I20260807 18:34:03.580067 21789 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:34:03.580312 21789 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:34:03.582424 21789 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786127579900523-11321-0/minicluster-data/master-0/data/instance:
uuid: "a7a92ecc7aca4f90a38055f156385c2a"
format_stamp: "Formatted at 2026-08-07 18:34:03 on dist-test-slave-6k22"
server_key: "2b8d7b738ce2660d92bf16928cbf7fa2"
server_key_iv: "dbf3c7e522b9f9d3deff44df30d7bfd6"
server_key_version: "encryptionkey@0"
I20260807 18:34:03.582998 21789 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786127579900523-11321-0/minicluster-data/master-0/wal/instance:
uuid: "a7a92ecc7aca4f90a38055f156385c2a"
format_stamp: "Formatted at 2026-08-07 18:34:03 on dist-test-slave-6k22"
server_key: "2b8d7b738ce2660d92bf16928cbf7fa2"
server_key_iv: "dbf3c7e522b9f9d3deff44df30d7bfd6"
server_key_version: "encryptionkey@0"
I20260807 18:34:03.586691 21789 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260807 18:34:03.589073 21805 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:34:03.590234 21789 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 18:34:03.590369 21789 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786127579900523-11321-0/minicluster-data/master-0/data,/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786127579900523-11321-0/minicluster-data/master-0/wal
uuid: "a7a92ecc7aca4f90a38055f156385c2a"
format_stamp: "Formatted at 2026-08-07 18:34:03 on dist-test-slave-6k22"
server_key: "2b8d7b738ce2660d92bf16928cbf7fa2"
server_key_iv: "dbf3c7e522b9f9d3deff44df30d7bfd6"
server_key_version: "encryptionkey@0"
I20260807 18:34:03.590476 21789 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786127579900523-11321-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786127579900523-11321-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786127579900523-11321-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:34:03.595054 21789 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:34:03.595705 21789 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:34:03.595880 21789 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:34:03.602806 21789 rpc_server.cc:307] RPC server started. Bound to: 127.11.14.126:43697
I20260807 18:34:03.602823 21862 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.14.126:43697 every 8 connection(s)
I20260807 18:34:03.603899 21789 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786127579900523-11321-0/minicluster-data/master-0/data/info.pb
I20260807 18:34:03.604326 11321 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkz56XN/build/debug/bin/kudu as pid 21789
I20260807 18:34:03.604455 11321 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786127579900523-11321-0/minicluster-data/master-0/wal/instance
I20260807 18:34:03.604732 11321 external_mini_cluster.cc:1428] Setting key 01a75159a6c84c27b8953cb8a6955588
I20260807 18:34:03.606897 21863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:34:03.610821 21866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:34:03.606086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60056 (local address 127.11.14.126:43697)
0807 18:34:03.607189 (+ 1103us) server_negotiation.cc:206] Beginning negotiation
0807 18:34:03.607212 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0807 18:34:03.607258 (+ 46us) server_negotiation.cc:407] Connection header received
0807 18:34:03.607993 (+ 735us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:34:03.608012 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:34:03.608368 (+ 356us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:34:03.608706 (+ 338us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:34:03.609531 (+ 825us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:34:03.609536 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:34:03.609548 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:34:03.609558 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:34:03.609614 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:34:03.609630 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:34:03.609777 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0807 18:34:03.609860 (+ 83us) server_negotiation.cc:299] Negotiation successful
0807 18:34:03.610094 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":673,"thread_start_us":543,"threads_started":1}
I20260807 18:34:03.613123 21863 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a7a92ecc7aca4f90a38055f156385c2a: Bootstrap starting.
I20260807 18:34:03.615222 21863 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a7a92ecc7aca4f90a38055f156385c2a: Neither blocks nor log segments found. Creating new log.
I20260807 18:34:03.616001 21863 log.cc:824] T 00000000000000000000000000000000 P a7a92ecc7aca4f90a38055f156385c2a: Log is configured to *not* fsync() on all Append() calls
I20260807 18:34:03.617946 21863 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a7a92ecc7aca4f90a38055f156385c2a: No bootstrap required, opened a new log
I20260807 18:34:03.621309 21863 raft_consensus.cc:348] T 00000000000000000000000000000000 P a7a92ecc7aca4f90a38055f156385c2a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7a92ecc7aca4f90a38055f156385c2a" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 43697 } }
I20260807 18:34:03.621536 21863 raft_consensus.cc:374] T 00000000000000000000000000000000 P a7a92ecc7aca4f90a38055f156385c2a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:34:03.621620 21863 raft_consensus.cc:729] T 00000000000000000000000000000000 P a7a92ecc7aca4f90a38055f156385c2a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a7a92ecc7aca4f90a38055f156385c2a, State: Initialized, Role: FOLLOWER
I20260807 18:34:03.622069 21863 consensus_queue.cc:260] T 00000000000000000000000000000000 P a7a92ecc7aca4f90a38055f156385c2a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7a92ecc7aca4f90a38055f156385c2a" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 43697 } }
I20260807 18:34:03.622216 21863 raft_consensus.cc:388] T 00000000000000000000000000000000 P a7a92ecc7aca4f90a38055f156385c2a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:34:03.622298 21863 raft_consensus.cc:482] T 00000000000000000000000000000000 P a7a92ecc7aca4f90a38055f156385c2a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:34:03.622397 21863 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a7a92ecc7aca4f90a38055f156385c2a [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:34:03.623220 21863 raft_consensus.cc:504] T 00000000000000000000000000000000 P a7a92ecc7aca4f90a38055f156385c2a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7a92ecc7aca4f90a38055f156385c2a" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 43697 } }
I20260807 18:34:03.623615 21863 leader_election.cc:302] T 00000000000000000000000000000000 P a7a92ecc7aca4f90a38055f156385c2a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a7a92ecc7aca4f90a38055f156385c2a; no voters:
I20260807 18:34:03.623945 21863 leader_election.cc:288] T 00000000000000000000000000000000 P a7a92ecc7aca4f90a38055f156385c2a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:34:03.624058 21868 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a7a92ecc7aca4f90a38055f156385c2a [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:34:03.624279 21868 raft_consensus.cc:686] T 00000000000000000000000000000000 P a7a92ecc7aca4f90a38055f156385c2a [term 1 LEADER]: Becoming Leader. State: Replica: a7a92ecc7aca4f90a38055f156385c2a, State: Running, Role: LEADER
I20260807 18:34:03.624583 21868 consensus_queue.cc:237] T 00000000000000000000000000000000 P a7a92ecc7aca4f90a38055f156385c2a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7a92ecc7aca4f90a38055f156385c2a" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 43697 } }
I20260807 18:34:03.625130 21863 sys_catalog.cc:564] T 00000000000000000000000000000000 P a7a92ecc7aca4f90a38055f156385c2a [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:34:03.626242 21870 sys_catalog.cc:455] T 00000000000000000000000000000000 P a7a92ecc7aca4f90a38055f156385c2a [sys.catalog]: SysCatalogTable state changed. Reason: New leader a7a92ecc7aca4f90a38055f156385c2a. Latest consensus state: current_term: 1 leader_uuid: "a7a92ecc7aca4f90a38055f156385c2a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7a92ecc7aca4f90a38055f156385c2a" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 43697 } } }
I20260807 18:34:03.626370 21870 sys_catalog.cc:458] T 00000000000000000000000000000000 P a7a92ecc7aca4f90a38055f156385c2a [sys.catalog]: This master's current role is: LEADER
I20260807 18:34:03.626652 21869 sys_catalog.cc:455] T 00000000000000000000000000000000 P a7a92ecc7aca4f90a38055f156385c2a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a7a92ecc7aca4f90a38055f156385c2a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7a92ecc7aca4f90a38055f156385c2a" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 43697 } } }
I20260807 18:34:03.626745 21869 sys_catalog.cc:458] T 00000000000000000000000000000000 P a7a92ecc7aca4f90a38055f156385c2a [sys.catalog]: This master's current role is: LEADER
I20260807 18:34:03.626865 21880 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:34:03.630009 21880 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:34:03.635545 21880 catalog_manager.cc:1347] Generated new cluster ID: 2d9e26caa2b04a51ac0896969cd0eb4b
I20260807 18:34:03.635656 21880 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:34:03.664354 21880 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:34:03.664745 21880 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:34:03.674044 21880 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a7a92ecc7aca4f90a38055f156385c2a: Generated new TSK 0
I20260807 18:34:03.674880 21880 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:34:03.679824 11321 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkz56XN/build/debug/bin/kudu
/tmp/dist-test-taskkz56XN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786127579900523-11321-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786127579900523-11321-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786127579900523-11321-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786127579900523-11321-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.14.65:0
--local_ip_for_outbound_sockets=127.11.14.65
--webserver_interface=127.11.14.65
--webserver_port=0
--tserver_master_addrs=127.11.14.126:43697
--builtin_ntp_servers=127.11.14.84:43109
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260807 18:34:03.797114 21891 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:34:03.797415 21891 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:34:03.797498 21891 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:34:03.801543 21891 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:34:03.801620 21891 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:34:03.801716 21891 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.14.65
I20260807 18:34:03.806475 21891 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.14.84:43109
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786127579900523-11321-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786127579900523-11321-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.14.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786127579900523-11321-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.14.65
--webserver_port=0
--tserver_master_addrs=127.11.14.126:43697
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21891
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.14.65
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786127579900523-11321-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:34:03.807607 21891 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:34:03.809005 21891 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:34:03.818706 21896 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:34:03.818749 21897 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:34:03.818899 21891 server_base.cc:1034] running on GCE node
W20260807 18:34:03.818758 21899 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:34:03.819384 21891 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:34:03.820088 21891 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:34:03.821272 21891 hybrid_clock.cc:648] HybridClock initialized: now 1786127643821248 us; error 33 us; skew 500 ppm
I20260807 18:34:03.823138 21891 webserver.cc:460] Webserver started at http://127.11.14.65:44993/ using document root <none> and password file <none>
I20260807 18:34:03.823810 21891 fs_manager.cc:359] Metadata directory not provided
I20260807 18:34:03.823921 21891 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:34:03.824160 21891 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:34:03.826390 21891 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786127579900523-11321-0/minicluster-data/ts-0/data/instance:
uuid: "7cf5bbccbdad4167a5707ce44b76d8d4"
format_stamp: "Formatted at 2026-08-07 18:34:03 on dist-test-slave-6k22"
server_key: "a53df6f4e4123f43d4094384a6dc448a"
server_key_iv: "65e3a37596445ea311d400441590c046"
server_key_version: "encryptionkey@0"
I20260807 18:34:03.827010 21891 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786127579900523-11321-0/minicluster-data/ts-0/wal/instance:
uuid: "7cf5bbccbdad4167a5707ce44b76d8d4"
format_stamp: "Formatted at 2026-08-07 18:34:03 on dist-test-slave-6k22"
server_key: "a53df6f4e4123f43d4094384a6dc448a"
server_key_iv: "65e3a37596445ea311d400441590c046"
server_key_version: "encryptionkey@0"
I20260807 18:34:03.830852 21891 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 18:34:03.833462 21905 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:34:03.834669 21891 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 18:34:03.834795 21891 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786127579900523-11321-0/minicluster-data/ts-0/data,/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786127579900523-11321-0/minicluster-data/ts-0/wal
uuid: "7cf5bbccbdad4167a5707ce44b76d8d4"
format_stamp: "Formatted at 2026-08-07 18:34:03 on dist-test-slave-6k22"
server_key: "a53df6f4e4123f43d4094384a6dc448a"
server_key_iv: "65e3a37596445ea311d400441590c046"
server_key_version: "encryptionkey@0"
I20260807 18:34:03.834908 21891 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786127579900523-11321-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786127579900523-11321-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786127579900523-11321-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:34:03.839658 21891 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:34:03.840256 21891 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:34:03.840443 21891 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:34:03.840996 21891 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:34:03.841893 21891 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:34:03.841940 21891 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:34:03.842003 21891 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:34:03.842034 21891 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:34:03.851486 21891 rpc_server.cc:307] RPC server started. Bound to: 127.11.14.65:44061
I20260807 18:34:03.851507 22026 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.14.65:44061 every 8 connection(s)
I20260807 18:34:03.852406 21891 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786127579900523-11321-0/minicluster-data/ts-0/data/info.pb
I20260807 18:34:03.857014 11321 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkz56XN/build/debug/bin/kudu as pid 21891
I20260807 18:34:03.857141 11321 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786127579900523-11321-0/minicluster-data/ts-0/wal/instance
I20260807 18:34:03.857474 11321 external_mini_cluster.cc:1428] Setting key 8f17dcdece381569fe2369ae8cf66ea0
I20260807 18:34:03.858704 21866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:34:03.854183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.65:40007 (local address 127.11.14.126:43697)
0807 18:34:03.854326 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0807 18:34:03.854337 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 18:34:03.855357 (+ 1020us) server_negotiation.cc:407] Connection header received
0807 18:34:03.856886 (+ 1529us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:34:03.856889 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:34:03.856935 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:34:03.857019 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:34:03.858239 (+ 1220us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:34:03.858242 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:34:03.858244 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:34:03.858256 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:34:03.858292 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:34:03.858301 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:34:03.858393 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0807 18:34:03.858565 (+ 172us) server_negotiation.cc:299] Negotiation successful
0807 18:34:03.858601 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260807 18:34:03.859683 22029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:34:03.854580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:43697 (local address 127.11.14.65:40007)
0807 18:34:03.855215 (+ 635us) negotiation.cc:107] Waiting for socket to connect
0807 18:34:03.855256 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 18:34:03.856623 (+ 1367us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:34:03.857227 (+ 604us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:34:03.857236 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:34:03.857368 (+ 132us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:34:03.857938 (+ 570us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 18:34:03.857940 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:34:03.857980 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 18:34:03.858069 (+ 89us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:34:03.858401 (+ 332us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:34:03.858412 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:34:03.858418 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 18:34:03.858591 (+ 173us) client_negotiation.cc:240] Negotiation successful
0807 18:34:03.858822 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":111,"threads_started":1}
I20260807 18:34:03.859687 11321 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkz56XN/build/debug/bin/kudu
/tmp/dist-test-taskkz56XN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786127579900523-11321-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786127579900523-11321-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786127579900523-11321-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786127579900523-11321-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.14.66:0
--local_ip_for_outbound_sockets=127.11.14.66
--webserver_interface=127.11.14.66
--webserver_port=0
--tserver_master_addrs=127.11.14.126:43697
--builtin_ntp_servers=127.11.14.84:43109
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260807 18:34:03.860836 22027 heartbeater.cc:347] Connected to a master server at 127.11.14.126:43697
I20260807 18:34:03.861080 22027 heartbeater.cc:464] Registering TS with master...
I20260807 18:34:03.861557 22027 heartbeater.cc:511] Master 127.11.14.126:43697 requested a full tablet report, sending...
I20260807 18:34:03.863010 21823 ts_manager.cc:194] Registered new tserver with Master: 7cf5bbccbdad4167a5707ce44b76d8d4 (127.11.14.65:44061)
W20260807 18:34:03.974890 22030 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:34:03.975188 22030 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:34:03.975266 22030 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:34:03.979245 22030 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:34:03.979323 22030 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:34:03.979410 22030 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.14.66
I20260807 18:34:03.984179 22030 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.14.84:43109
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786127579900523-11321-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786127579900523-11321-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.14.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786127579900523-11321-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.14.66
--webserver_port=0
--tserver_master_addrs=127.11.14.126:43697
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22030
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.14.66
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786127579900523-11321-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:34:03.985229 22030 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:34:03.986583 22030 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:34:03.996965 22038 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:34:03.996965 22036 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:34:03.997179 22030 server_base.cc:1034] running on GCE node
W20260807 18:34:03.997267 22035 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:34:03.997684 22030 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:34:03.998318 22030 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:34:03.999523 22030 hybrid_clock.cc:648] HybridClock initialized: now 1786127643999498 us; error 37 us; skew 500 ppm
I20260807 18:34:04.001327 22030 webserver.cc:460] Webserver started at http://127.11.14.66:44925/ using document root <none> and password file <none>
I20260807 18:34:04.001919 22030 fs_manager.cc:359] Metadata directory not provided
I20260807 18:34:04.002012 22030 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:34:04.002256 22030 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:34:04.004387 22030 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786127579900523-11321-0/minicluster-data/ts-1/data/instance:
uuid: "6f1022939dbe455eb997935a073d93a1"
format_stamp: "Formatted at 2026-08-07 18:34:03 on dist-test-slave-6k22"
server_key: "27191bb01f813a968af886be4447a878"
server_key_iv: "6f504ffbbf89b16436e15bc24afd20a7"
server_key_version: "encryptionkey@0"
I20260807 18:34:04.004963 22030 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786127579900523-11321-0/minicluster-data/ts-1/wal/instance:
uuid: "6f1022939dbe455eb997935a073d93a1"
format_stamp: "Formatted at 2026-08-07 18:34:03 on dist-test-slave-6k22"
server_key: "27191bb01f813a968af886be4447a878"
server_key_iv: "6f504ffbbf89b16436e15bc24afd20a7"
server_key_version: "encryptionkey@0"
I20260807 18:34:04.008777 22030 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 18:34:04.011270 22045 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:34:04.012507 22030 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 18:34:04.012640 22030 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786127579900523-11321-0/minicluster-data/ts-1/data,/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786127579900523-11321-0/minicluster-data/ts-1/wal
uuid: "6f1022939dbe455eb997935a073d93a1"
format_stamp: "Formatted at 2026-08-07 18:34:03 on dist-test-slave-6k22"
server_key: "27191bb01f813a968af886be4447a878"
server_key_iv: "6f504ffbbf89b16436e15bc24afd20a7"
server_key_version: "encryptionkey@0"
I20260807 18:34:04.012750 22030 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786127579900523-11321-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786127579900523-11321-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786127579900523-11321-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:34:04.017283 22030 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:34:04.017868 22030 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:34:04.018038 22030 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:34:04.018601 22030 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:34:04.019503 22030 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:34:04.019780 22030 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:34:04.019898 22030 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:34:04.019958 22030 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:34:04.030246 22030 rpc_server.cc:307] RPC server started. Bound to: 127.11.14.66:36949
I20260807 18:34:04.030264 22164 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.14.66:36949 every 8 connection(s)
I20260807 18:34:04.031253 22030 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786127579900523-11321-0/minicluster-data/ts-1/data/info.pb
I20260807 18:34:04.037197 11321 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkz56XN/build/debug/bin/kudu as pid 22030
I20260807 18:34:04.037352 11321 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786127579900523-11321-0/minicluster-data/ts-1/wal/instance
I20260807 18:34:04.037711 11321 external_mini_cluster.cc:1428] Setting key 0d33319a35ab10bca0d2ac946e6d8252
I20260807 18:34:04.038159 21866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:34:04.033374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.66:52857 (local address 127.11.14.126:43697)
0807 18:34:04.033551 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0807 18:34:04.033556 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:34:04.034856 (+ 1300us) server_negotiation.cc:407] Connection header received
0807 18:34:04.036251 (+ 1395us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:34:04.036254 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:34:04.036310 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:34:04.036392 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:34:04.037613 (+ 1221us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:34:04.037616 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:34:04.037619 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:34:04.037632 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:34:04.037656 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:34:04.037671 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:34:04.037813 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0807 18:34:04.037983 (+ 170us) server_negotiation.cc:299] Negotiation successful
0807 18:34:04.038027 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260807 18:34:04.039228 22167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:34:04.033953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:43697 (local address 127.11.14.66:52857)
0807 18:34:04.034676 (+ 723us) negotiation.cc:107] Waiting for socket to connect
0807 18:34:04.034718 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 18:34:04.036041 (+ 1323us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:34:04.036576 (+ 535us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:34:04.036584 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:34:04.036673 (+ 89us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:34:04.037272 (+ 599us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 18:34:04.037274 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:34:04.037346 (+ 72us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 18:34:04.037442 (+ 96us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:34:04.037799 (+ 357us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:34:04.037810 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:34:04.037817 (+ 7us) client_negotiation.cc:769] Sending connection context
0807 18:34:04.038025 (+ 208us) client_negotiation.cc:240] Negotiation successful
0807 18:34:04.038291 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":307,"thread_start_us":165,"threads_started":1}
I20260807 18:34:04.040179 11321 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkz56XN/build/debug/bin/kudu
/tmp/dist-test-taskkz56XN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786127579900523-11321-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786127579900523-11321-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786127579900523-11321-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786127579900523-11321-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.14.67:0
--local_ip_for_outbound_sockets=127.11.14.67
--webserver_interface=127.11.14.67
--webserver_port=0
--tserver_master_addrs=127.11.14.126:43697
--builtin_ntp_servers=127.11.14.84:43109
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260807 18:34:04.040457 22165 heartbeater.cc:347] Connected to a master server at 127.11.14.126:43697
I20260807 18:34:04.040700 22165 heartbeater.cc:464] Registering TS with master...
I20260807 18:34:04.041180 22165 heartbeater.cc:511] Master 127.11.14.126:43697 requested a full tablet report, sending...
I20260807 18:34:04.042397 21823 ts_manager.cc:194] Registered new tserver with Master: 6f1022939dbe455eb997935a073d93a1 (127.11.14.66:36949)
W20260807 18:34:04.155275 22168 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:34:04.155534 22168 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:34:04.155602 22168 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:34:04.159677 22168 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:34:04.159727 22168 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:34:04.159787 22168 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.14.67
I20260807 18:34:04.164462 22168 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.14.84:43109
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786127579900523-11321-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786127579900523-11321-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.14.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786127579900523-11321-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.14.67
--webserver_port=0
--tserver_master_addrs=127.11.14.126:43697
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22168
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.14.67
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786127579900523-11321-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:34:04.165512 22168 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:34:04.166889 22168 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:34:04.177582 22173 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:34:04.177663 22174 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:34:04.177675 22176 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:34:04.178732 22168 server_base.cc:1034] running on GCE node
I20260807 18:34:04.179261 22168 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:34:04.179991 22168 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:34:04.181188 22168 hybrid_clock.cc:648] HybridClock initialized: now 1786127644181172 us; error 28 us; skew 500 ppm
I20260807 18:34:04.183060 22168 webserver.cc:460] Webserver started at http://127.11.14.67:40385/ using document root <none> and password file <none>
I20260807 18:34:04.183694 22168 fs_manager.cc:359] Metadata directory not provided
I20260807 18:34:04.183794 22168 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:34:04.184024 22168 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:34:04.186081 22168 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786127579900523-11321-0/minicluster-data/ts-2/data/instance:
uuid: "e6170a0ce5cb48b481b7d4b8f68b1696"
format_stamp: "Formatted at 2026-08-07 18:34:04 on dist-test-slave-6k22"
server_key: "30112d881a1633eaf8ddb07b7098ed58"
server_key_iv: "78a99dd30a98cc5b9ffa1c4868a4ae3d"
server_key_version: "encryptionkey@0"
I20260807 18:34:04.186657 22168 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786127579900523-11321-0/minicluster-data/ts-2/wal/instance:
uuid: "e6170a0ce5cb48b481b7d4b8f68b1696"
format_stamp: "Formatted at 2026-08-07 18:34:04 on dist-test-slave-6k22"
server_key: "30112d881a1633eaf8ddb07b7098ed58"
server_key_iv: "78a99dd30a98cc5b9ffa1c4868a4ae3d"
server_key_version: "encryptionkey@0"
I20260807 18:34:04.190371 22168 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.003s
I20260807 18:34:04.192953 22183 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:34:04.194262 22168 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 18:34:04.194386 22168 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786127579900523-11321-0/minicluster-data/ts-2/data,/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786127579900523-11321-0/minicluster-data/ts-2/wal
uuid: "e6170a0ce5cb48b481b7d4b8f68b1696"
format_stamp: "Formatted at 2026-08-07 18:34:04 on dist-test-slave-6k22"
server_key: "30112d881a1633eaf8ddb07b7098ed58"
server_key_iv: "78a99dd30a98cc5b9ffa1c4868a4ae3d"
server_key_version: "encryptionkey@0"
I20260807 18:34:04.194507 22168 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786127579900523-11321-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786127579900523-11321-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786127579900523-11321-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:34:04.198978 22168 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:34:04.199628 22168 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:34:04.199800 22168 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:34:04.200340 22168 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:34:04.201227 22168 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:34:04.201303 22168 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:34:04.201371 22168 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:34:04.201417 22168 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:34:04.209995 22168 rpc_server.cc:307] RPC server started. Bound to: 127.11.14.67:37083
I20260807 18:34:04.210021 22305 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.14.67:37083 every 8 connection(s)
I20260807 18:34:04.210858 22168 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786127579900523-11321-0/minicluster-data/ts-2/data/info.pb
I20260807 18:34:04.216914 21866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:34:04.212682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.67:39557 (local address 127.11.14.126:43697)
0807 18:34:04.212825 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0807 18:34:04.212829 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:34:04.213862 (+ 1033us) server_negotiation.cc:407] Connection header received
0807 18:34:04.215202 (+ 1340us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:34:04.215205 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:34:04.215250 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:34:04.215357 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:34:04.216456 (+ 1099us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:34:04.216460 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:34:04.216463 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:34:04.216475 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:34:04.216503 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:34:04.216512 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:34:04.216614 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0807 18:34:04.216767 (+ 153us) server_negotiation.cc:299] Negotiation successful
0807 18:34:04.216807 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260807 18:34:04.217839 22308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:34:04.213065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:43697 (local address 127.11.14.67:39557)
0807 18:34:04.213704 (+ 639us) negotiation.cc:107] Waiting for socket to connect
0807 18:34:04.213744 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0807 18:34:04.214974 (+ 1230us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:34:04.215524 (+ 550us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:34:04.215530 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:34:04.215669 (+ 139us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:34:04.216199 (+ 530us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 18:34:04.216200 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:34:04.216239 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 18:34:04.216310 (+ 71us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:34:04.216629 (+ 319us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:34:04.216638 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:34:04.216644 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 18:34:04.216791 (+ 147us) client_negotiation.cc:240] Negotiation successful
0807 18:34:04.216983 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":119,"threads_started":1}
I20260807 18:34:04.217937 11321 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkz56XN/build/debug/bin/kudu as pid 22168
I20260807 18:34:04.218056 11321 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786127579900523-11321-0/minicluster-data/ts-2/wal/instance
I20260807 18:34:04.218441 11321 external_mini_cluster.cc:1428] Setting key 1a3b07a2303c19c0d2f79a515ab2c772
I20260807 18:34:04.218998 22306 heartbeater.cc:347] Connected to a master server at 127.11.14.126:43697
I20260807 18:34:04.219223 22306 heartbeater.cc:464] Registering TS with master...
I20260807 18:34:04.219707 22306 heartbeater.cc:511] Master 127.11.14.126:43697 requested a full tablet report, sending...
I20260807 18:34:04.220676 21823 ts_manager.cc:194] Registered new tserver with Master: e6170a0ce5cb48b481b7d4b8f68b1696 (127.11.14.67:37083)
I20260807 18:34:04.232115 11321 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260807 18:34:04.234911 21866 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:34:04.234023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60066 (local address 127.11.14.126:43697)
0807 18:34:04.234174 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0807 18:34:04.234179 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:34:04.234325 (+ 146us) server_negotiation.cc:407] Connection header received
0807 18:34:04.234425 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:34:04.234427 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:34:04.234458 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:34:04.234515 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:34:04.234762 (+ 247us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60066: BlockingRecv error: recv got EOF from 127.0.0.1:60066 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
I20260807 18:34:04.235035 11321 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkz56XN/build/debug/bin/kudu with pid 21891
I20260807 18:34:04.246225 11321 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkz56XN/build/debug/bin/kudu with pid 22030
I20260807 18:34:04.253290 11321 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkz56XN/build/debug/bin/kudu with pid 22168
I20260807 18:34:04.261726 11321 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkz56XN/build/debug/bin/kudu with pid 21789
2026-08-07T18:34:04Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (855 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127579900523-11321-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:34:04 dist-test-slave-6k22 krb5kdc[22316](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:34:04 dist-test-slave-6k22 krb5kdc[22316](info): set up 2 sockets
Aug 07 18:34:04 dist-test-slave-6k22 krb5kdc[22316](info): commencing operation
krb5kdc: starting...
W20260807 18:34:05.372515 11321 mini_kdc.cc:121] Time spent starting KDC: real 1.088s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127579900523-11321-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127579900523-11321-0/krb5kdc/test-user.keytab.
Aug 07 18:34:05 dist-test-slave-6k22 krb5kdc[22316](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127645, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:34:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:34:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.14.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.14.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.14.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.126.keytab.
Entry for principal kudu/127.11.14.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.126.keytab.
I20260807 18:34:05.501996 11321 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkz56XN/build/debug/bin/kudu
/tmp/dist-test-taskkz56XN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127579900523-11321-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127579900523-11321-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127579900523-11321-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127579900523-11321-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.14.126:35017
--webserver_interface=127.11.14.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.126.keytab
--principal=kudu/127.11.14.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.14.84:37385
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.14.126:35017
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127579900523-11321-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127579900523-11321-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127579900523-11321-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:34:05.617340 22329 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:34:05.617609 22329 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:34:05.617653 22329 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:34:05.621658 22329 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:34:05.621723 22329 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:34:05.621745 22329 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:34:05.621765 22329 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:34:05.621784 22329 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:34:05.626906 22329 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.14.84:37385
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127579900523-11321-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127579900523-11321-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.14.126:35017
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.126.keytab
--principal=kudu/127.11.14.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.14.126:35017
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127579900523-11321-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.14.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22329
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127579900523-11321-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:34:05.627938 22329 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:34:05.628924 22329 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:34:05.637652 22335 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:34:05.637668 22334 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:34:05.637878 22329 server_base.cc:1034] running on GCE node
W20260807 18:34:05.637656 22337 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:34:05.638667 22329 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:34:05.639936 22329 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:34:05.641160 22329 hybrid_clock.cc:648] HybridClock initialized: now 1786127645641124 us; error 46 us; skew 500 ppm
Aug 07 18:34:05 dist-test-slave-6k22 krb5kdc[22316](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127645, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:34:05.644021 22329 init.cc:377] Logged in from keytab as kudu/127.11.14.126@KRBTEST.COM (short username kudu)
I20260807 18:34:05.644997 22329 webserver.cc:460] Webserver started at http://127.11.14.126:43439/ using document root <none> and password file <none>
I20260807 18:34:05.645591 22329 fs_manager.cc:359] Metadata directory not provided
I20260807 18:34:05.645682 22329 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:34:05.645923 22329 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:34:05.648100 22329 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127579900523-11321-0/minicluster-data/master-0/data/instance:
uuid: "087469ff056948aba87aecb0a72c76d2"
format_stamp: "Formatted at 2026-08-07 18:34:05 on dist-test-slave-6k22"
server_key: "dc9a49cc61284750a81e601bfed4c19d"
server_key_iv: "a23c540c83b3531bad46958bc75bc691"
server_key_version: "encryptionkey@0"
I20260807 18:34:05.648699 22329 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127579900523-11321-0/minicluster-data/master-0/wal/instance:
uuid: "087469ff056948aba87aecb0a72c76d2"
format_stamp: "Formatted at 2026-08-07 18:34:05 on dist-test-slave-6k22"
server_key: "dc9a49cc61284750a81e601bfed4c19d"
server_key_iv: "a23c540c83b3531bad46958bc75bc691"
server_key_version: "encryptionkey@0"
I20260807 18:34:05.652541 22329 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 18:34:05.654961 22345 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:34:05.656318 22329 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 18:34:05.656473 22329 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127579900523-11321-0/minicluster-data/master-0/data,/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127579900523-11321-0/minicluster-data/master-0/wal
uuid: "087469ff056948aba87aecb0a72c76d2"
format_stamp: "Formatted at 2026-08-07 18:34:05 on dist-test-slave-6k22"
server_key: "dc9a49cc61284750a81e601bfed4c19d"
server_key_iv: "a23c540c83b3531bad46958bc75bc691"
server_key_version: "encryptionkey@0"
I20260807 18:34:05.656582 22329 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127579900523-11321-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127579900523-11321-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127579900523-11321-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:34:05.688262 22329 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:34:05.691375 22329 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:34:05.691632 22329 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:34:05.699285 22329 rpc_server.cc:307] RPC server started. Bound to: 127.11.14.126:35017
I20260807 18:34:05.699618 22401 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.14.126:35017 every 8 connection(s)
I20260807 18:34:05.700376 22329 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127579900523-11321-0/minicluster-data/master-0/data/info.pb
I20260807 18:34:05.703181 22402 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:34:05.708686 11321 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkz56XN/build/debug/bin/kudu as pid 22329
I20260807 18:34:05.708810 11321 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127579900523-11321-0/minicluster-data/master-0/wal/instance
I20260807 18:34:05.708895 22402 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 087469ff056948aba87aecb0a72c76d2: Bootstrap starting.
I20260807 18:34:05.709132 11321 external_mini_cluster.cc:1428] Setting key f6b063e64b026d7a82344a31d4feebb7
I20260807 18:34:05.711511 22402 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 087469ff056948aba87aecb0a72c76d2: Neither blocks nor log segments found. Creating new log.
I20260807 18:34:05.712323 22402 log.cc:824] T 00000000000000000000000000000000 P 087469ff056948aba87aecb0a72c76d2: Log is configured to *not* fsync() on all Append() calls
I20260807 18:34:05.714267 22402 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 087469ff056948aba87aecb0a72c76d2: No bootstrap required, opened a new log
Aug 07 18:34:05 dist-test-slave-6k22 krb5kdc[22316](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127645, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.14.126@KRBTEST.COM
I20260807 18:34:05.718719 22402 raft_consensus.cc:348] T 00000000000000000000000000000000 P 087469ff056948aba87aecb0a72c76d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "087469ff056948aba87aecb0a72c76d2" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 35017 } }
I20260807 18:34:05.718976 22402 raft_consensus.cc:374] T 00000000000000000000000000000000 P 087469ff056948aba87aecb0a72c76d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:34:05.719050 22402 raft_consensus.cc:729] T 00000000000000000000000000000000 P 087469ff056948aba87aecb0a72c76d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 087469ff056948aba87aecb0a72c76d2, State: Initialized, Role: FOLLOWER
I20260807 18:34:05.719590 22402 consensus_queue.cc:260] T 00000000000000000000000000000000 P 087469ff056948aba87aecb0a72c76d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "087469ff056948aba87aecb0a72c76d2" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 35017 } }
I20260807 18:34:05.719791 22402 raft_consensus.cc:388] T 00000000000000000000000000000000 P 087469ff056948aba87aecb0a72c76d2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:34:05.719883 22402 raft_consensus.cc:482] T 00000000000000000000000000000000 P 087469ff056948aba87aecb0a72c76d2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:34:05.719985 22402 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 087469ff056948aba87aecb0a72c76d2 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:34:05.720886 22402 raft_consensus.cc:504] T 00000000000000000000000000000000 P 087469ff056948aba87aecb0a72c76d2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "087469ff056948aba87aecb0a72c76d2" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 35017 } }
I20260807 18:34:05.721235 22402 leader_election.cc:302] T 00000000000000000000000000000000 P 087469ff056948aba87aecb0a72c76d2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 087469ff056948aba87aecb0a72c76d2; no voters:
I20260807 18:34:05.721578 22402 leader_election.cc:288] T 00000000000000000000000000000000 P 087469ff056948aba87aecb0a72c76d2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:34:05.721645 22407 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 087469ff056948aba87aecb0a72c76d2 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:34:05.721881 22407 raft_consensus.cc:686] T 00000000000000000000000000000000 P 087469ff056948aba87aecb0a72c76d2 [term 1 LEADER]: Becoming Leader. State: Replica: 087469ff056948aba87aecb0a72c76d2, State: Running, Role: LEADER
I20260807 18:34:05.722187 22407 consensus_queue.cc:237] T 00000000000000000000000000000000 P 087469ff056948aba87aecb0a72c76d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "087469ff056948aba87aecb0a72c76d2" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 35017 } }
I20260807 18:34:05.722949 22402 sys_catalog.cc:564] T 00000000000000000000000000000000 P 087469ff056948aba87aecb0a72c76d2 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:34:05.724315 22409 sys_catalog.cc:455] T 00000000000000000000000000000000 P 087469ff056948aba87aecb0a72c76d2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 087469ff056948aba87aecb0a72c76d2. Latest consensus state: current_term: 1 leader_uuid: "087469ff056948aba87aecb0a72c76d2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "087469ff056948aba87aecb0a72c76d2" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 35017 } } }
I20260807 18:34:05.724359 22408 sys_catalog.cc:455] T 00000000000000000000000000000000 P 087469ff056948aba87aecb0a72c76d2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "087469ff056948aba87aecb0a72c76d2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "087469ff056948aba87aecb0a72c76d2" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 35017 } } }
I20260807 18:34:05.724445 22409 sys_catalog.cc:458] T 00000000000000000000000000000000 P 087469ff056948aba87aecb0a72c76d2 [sys.catalog]: This master's current role is: LEADER
I20260807 18:34:05.724463 22408 sys_catalog.cc:458] T 00000000000000000000000000000000 P 087469ff056948aba87aecb0a72c76d2 [sys.catalog]: This master's current role is: LEADER
I20260807 18:34:05.724783 22405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:34:05.710481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43738 (local address 127.11.14.126:35017)
0807 18:34:05.710899 (+ 418us) server_negotiation.cc:206] Beginning negotiation
0807 18:34:05.710921 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0807 18:34:05.710958 (+ 37us) server_negotiation.cc:407] Connection header received
0807 18:34:05.711645 (+ 687us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:34:05.711670 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:34:05.712018 (+ 348us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:34:05.712297 (+ 279us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:34:05.713205 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:34:05.714183 (+ 978us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:34:05.715357 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:34:05.715875 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:34:05.715997 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:34:05.718927 (+ 2930us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:34:05.718933 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:34:05.718946 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:34:05.718974 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:34:05.721060 (+ 2086us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:34:05.721559 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:34:05.721566 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:34:05.721577 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:34:05.721743 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:34:05.722129 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:34:05.722133 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:34:05.722135 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:34:05.722636 (+ 501us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:34:05.722773 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0807 18:34:05.723308 (+ 535us) server_negotiation.cc:299] Negotiation successful
0807 18:34:05.723657 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":274,"thread_start_us":140,"threads_started":1}
I20260807 18:34:05.724910 22416 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:34:05.728233 22416 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:34:05.734076 22416 catalog_manager.cc:1347] Generated new cluster ID: e4c28fe17c5349609396fa21bfe5af09
I20260807 18:34:05.734143 22416 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:34:05.748579 22416 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:34:05.749853 22416 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:34:05.771909 22416 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 087469ff056948aba87aecb0a72c76d2: Generated new TSK 0
I20260807 18:34:05.772847 22416 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.14.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.14.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.14.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.65.keytab.
Entry for principal kudu/127.11.14.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.65.keytab.
I20260807 18:34:05.808369 11321 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkz56XN/build/debug/bin/kudu
/tmp/dist-test-taskkz56XN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127579900523-11321-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127579900523-11321-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127579900523-11321-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127579900523-11321-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.14.65:0
--local_ip_for_outbound_sockets=127.11.14.65
--webserver_interface=127.11.14.65
--webserver_port=0
--tserver_master_addrs=127.11.14.126:35017
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.65.keytab
--principal=kudu/127.11.14.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.14.84:37385
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127579900523-11321-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127579900523-11321-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127579900523-11321-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:34:05.925804 22428 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:34:05.926054 22428 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:34:05.926095 22428 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:34:05.930080 22428 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:34:05.930130 22428 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:34:05.930202 22428 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.14.65
I20260807 18:34:05.935343 22428 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.14.84:37385
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127579900523-11321-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127579900523-11321-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.65.keytab
--principal=kudu/127.11.14.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.14.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127579900523-11321-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.14.65
--webserver_port=0
--tserver_master_addrs=127.11.14.126:35017
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22428
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.14.65
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127579900523-11321-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:34:05.936400 22428 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:34:05.937520 22428 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 18:34:05.947623 22428 server_base.cc:1034] running on GCE node
W20260807 18:34:05.947530 22434 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:34:05.947513 22436 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:34:05.947513 22433 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:34:05.948215 22428 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:34:05.948845 22428 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:34:05.950038 22428 hybrid_clock.cc:648] HybridClock initialized: now 1786127645950026 us; error 30 us; skew 500 ppm
Aug 07 18:34:05 dist-test-slave-6k22 krb5kdc[22316](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127645, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:34:05.952848 22428 init.cc:377] Logged in from keytab as kudu/127.11.14.65@KRBTEST.COM (short username kudu)
I20260807 18:34:05.953691 22428 webserver.cc:460] Webserver started at http://127.11.14.65:38613/ using document root <none> and password file <none>
I20260807 18:34:05.954272 22428 fs_manager.cc:359] Metadata directory not provided
I20260807 18:34:05.954380 22428 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:34:05.954627 22428 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:34:05.956836 22428 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127579900523-11321-0/minicluster-data/ts-0/data/instance:
uuid: "ce6c17b71ecc4239ad183d7a58e48d13"
format_stamp: "Formatted at 2026-08-07 18:34:05 on dist-test-slave-6k22"
server_key: "b55c087729ef24465308b69535992885"
server_key_iv: "c1bca9a0e4938545deeb8dab6076c011"
server_key_version: "encryptionkey@0"
I20260807 18:34:05.957406 22428 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127579900523-11321-0/minicluster-data/ts-0/wal/instance:
uuid: "ce6c17b71ecc4239ad183d7a58e48d13"
format_stamp: "Formatted at 2026-08-07 18:34:05 on dist-test-slave-6k22"
server_key: "b55c087729ef24465308b69535992885"
server_key_iv: "c1bca9a0e4938545deeb8dab6076c011"
server_key_version: "encryptionkey@0"
I20260807 18:34:05.961642 22428 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 18:34:05.964077 22444 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:34:05.965277 22428 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 18:34:05.965456 22428 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127579900523-11321-0/minicluster-data/ts-0/data,/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127579900523-11321-0/minicluster-data/ts-0/wal
uuid: "ce6c17b71ecc4239ad183d7a58e48d13"
format_stamp: "Formatted at 2026-08-07 18:34:05 on dist-test-slave-6k22"
server_key: "b55c087729ef24465308b69535992885"
server_key_iv: "c1bca9a0e4938545deeb8dab6076c011"
server_key_version: "encryptionkey@0"
I20260807 18:34:05.965565 22428 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127579900523-11321-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127579900523-11321-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127579900523-11321-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:34:05.987782 22428 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:34:05.990844 22428 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:34:05.991079 22428 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:34:05.991804 22428 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:34:05.992992 22428 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:34:05.993067 22428 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:34:05.993144 22428 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:34:05.993198 22428 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:34:06.004209 22428 rpc_server.cc:307] RPC server started. Bound to: 127.11.14.65:42183
I20260807 18:34:06.004227 22563 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.14.65:42183 every 8 connection(s)
I20260807 18:34:06.005283 22428 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127579900523-11321-0/minicluster-data/ts-0/data/info.pb
I20260807 18:34:06.015074 11321 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkz56XN/build/debug/bin/kudu as pid 22428
I20260807 18:34:06.015213 11321 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127579900523-11321-0/minicluster-data/ts-0/wal/instance
Aug 07 18:34:06 dist-test-slave-6k22 krb5kdc[22316](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127645, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.65@KRBTEST.COM for kudu/127.11.14.126@KRBTEST.COM
I20260807 18:34:06.015511 11321 external_mini_cluster.cc:1428] Setting key 9f76225d03c50e6c79229cbf1fb302af
I20260807 18:34:06.020419 22405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:34:06.007226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.65:57975 (local address 127.11.14.126:35017)
0807 18:34:06.007418 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0807 18:34:06.007424 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:34:06.008448 (+ 1024us) server_negotiation.cc:407] Connection header received
0807 18:34:06.009823 (+ 1375us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:34:06.009826 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:34:06.009886 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:34:06.009981 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:34:06.011177 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:34:06.011947 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:34:06.013058 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:34:06.013328 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:34:06.013437 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:34:06.016284 (+ 2847us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:34:06.016292 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:34:06.016295 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:34:06.016320 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:34:06.018108 (+ 1788us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:34:06.018646 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:34:06.018652 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:34:06.018657 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:34:06.018761 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:34:06.019097 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:34:06.019104 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:34:06.019108 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:34:06.019371 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:34:06.019486 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0807 18:34:06.020185 (+ 699us) server_negotiation.cc:299] Negotiation successful
0807 18:34:06.020297 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260807 18:34:06.021680 22566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:34:06.007770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:35017 (local address 127.11.14.65:57975)
0807 18:34:06.008270 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0807 18:34:06.008327 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0807 18:34:06.009592 (+ 1265us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:34:06.010144 (+ 552us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:34:06.010151 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:34:06.010526 (+ 375us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:34:06.011011 (+ 485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:34:06.011025 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:34:06.012064 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:34:06.012067 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:34:06.012939 (+ 872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:34:06.012948 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:34:06.013492 (+ 544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:34:06.013495 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:34:06.013637 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:34:06.014241 (+ 604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:34:06.014258 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:34:06.016081 (+ 1823us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:34:06.018229 (+ 2148us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:34:06.018234 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:34:06.018247 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:34:06.018544 (+ 297us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:34:06.018869 (+ 325us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:34:06.018871 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:34:06.018872 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:34:06.019008 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:34:06.019503 (+ 495us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:34:06.019512 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:34:06.019905 (+ 393us) client_negotiation.cc:769] Sending connection context
0807 18:34:06.020149 (+ 244us) client_negotiation.cc:240] Negotiation successful
0807 18:34:06.020540 (+ 391us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":296,"thread_start_us":127,"threads_started":1}
I20260807 18:34:06.022974 22564 heartbeater.cc:347] Connected to a master server at 127.11.14.126:35017
I20260807 18:34:06.023267 22564 heartbeater.cc:464] Registering TS with master...
I20260807 18:34:06.023978 22564 heartbeater.cc:511] Master 127.11.14.126:35017 requested a full tablet report, sending...
I20260807 18:34:06.025611 22364 ts_manager.cc:194] Registered new tserver with Master: ce6c17b71ecc4239ad183d7a58e48d13 (127.11.14.65:42183)
I20260807 18:34:06.026744 22364 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.14.65@KRBTEST.COM'} at 127.11.14.65:57975
WARNING: no policy specified for kudu/127.11.14.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.14.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.14.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.66.keytab.
Entry for principal kudu/127.11.14.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.66.keytab.
I20260807 18:34:06.046092 11321 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkz56XN/build/debug/bin/kudu
/tmp/dist-test-taskkz56XN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127579900523-11321-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127579900523-11321-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127579900523-11321-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127579900523-11321-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.14.66:0
--local_ip_for_outbound_sockets=127.11.14.66
--webserver_interface=127.11.14.66
--webserver_port=0
--tserver_master_addrs=127.11.14.126:35017
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.66.keytab
--principal=kudu/127.11.14.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.14.84:37385
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127579900523-11321-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127579900523-11321-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127579900523-11321-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:34:06.161223 22569 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:34:06.161481 22569 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:34:06.161556 22569 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:34:06.165544 22569 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:34:06.165598 22569 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:34:06.165655 22569 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.14.66
I20260807 18:34:06.170874 22569 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.14.84:37385
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127579900523-11321-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127579900523-11321-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.66.keytab
--principal=kudu/127.11.14.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.14.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127579900523-11321-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.14.66
--webserver_port=0
--tserver_master_addrs=127.11.14.126:35017
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22569
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.14.66
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127579900523-11321-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:34:06.171938 22569 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:34:06.173187 22569 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:34:06.184602 22577 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:34:06.184628 22574 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:34:06.184612 22575 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:34:06.184914 22569 server_base.cc:1034] running on GCE node
I20260807 18:34:06.185382 22569 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:34:06.186005 22569 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:34:06.187173 22569 hybrid_clock.cc:648] HybridClock initialized: now 1786127646187145 us; error 33 us; skew 500 ppm
Aug 07 18:34:06 dist-test-slave-6k22 krb5kdc[22316](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127646, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:34:06.189949 22569 init.cc:377] Logged in from keytab as kudu/127.11.14.66@KRBTEST.COM (short username kudu)
I20260807 18:34:06.190850 22569 webserver.cc:460] Webserver started at http://127.11.14.66:33885/ using document root <none> and password file <none>
I20260807 18:34:06.191421 22569 fs_manager.cc:359] Metadata directory not provided
I20260807 18:34:06.191512 22569 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:34:06.191766 22569 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:34:06.193863 22569 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127579900523-11321-0/minicluster-data/ts-1/data/instance:
uuid: "64bdfa84d7cd495eb6282d2ba8d0cbff"
format_stamp: "Formatted at 2026-08-07 18:34:06 on dist-test-slave-6k22"
server_key: "e753ac7661957862f6a51c673c104662"
server_key_iv: "af62f64b0f24183a867c7a4a772041cd"
server_key_version: "encryptionkey@0"
I20260807 18:34:06.194437 22569 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127579900523-11321-0/minicluster-data/ts-1/wal/instance:
uuid: "64bdfa84d7cd495eb6282d2ba8d0cbff"
format_stamp: "Formatted at 2026-08-07 18:34:06 on dist-test-slave-6k22"
server_key: "e753ac7661957862f6a51c673c104662"
server_key_iv: "af62f64b0f24183a867c7a4a772041cd"
server_key_version: "encryptionkey@0"
I20260807 18:34:06.198194 22569 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260807 18:34:06.200706 22585 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:34:06.201915 22569 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 18:34:06.202049 22569 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127579900523-11321-0/minicluster-data/ts-1/data,/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127579900523-11321-0/minicluster-data/ts-1/wal
uuid: "64bdfa84d7cd495eb6282d2ba8d0cbff"
format_stamp: "Formatted at 2026-08-07 18:34:06 on dist-test-slave-6k22"
server_key: "e753ac7661957862f6a51c673c104662"
server_key_iv: "af62f64b0f24183a867c7a4a772041cd"
server_key_version: "encryptionkey@0"
I20260807 18:34:06.202157 22569 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127579900523-11321-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127579900523-11321-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127579900523-11321-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:34:06.218678 22569 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:34:06.221973 22569 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:34:06.222169 22569 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:34:06.222759 22569 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:34:06.223716 22569 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:34:06.223791 22569 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:34:06.223853 22569 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:34:06.223888 22569 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:34:06.233094 22569 rpc_server.cc:307] RPC server started. Bound to: 127.11.14.66:36445
I20260807 18:34:06.233117 22703 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.14.66:36445 every 8 connection(s)
I20260807 18:34:06.234025 22569 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127579900523-11321-0/minicluster-data/ts-1/data/info.pb
I20260807 18:34:06.243009 11321 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkz56XN/build/debug/bin/kudu as pid 22569
I20260807 18:34:06.243135 11321 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127579900523-11321-0/minicluster-data/ts-1/wal/instance
Aug 07 18:34:06 dist-test-slave-6k22 krb5kdc[22316](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127646, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.66@KRBTEST.COM for kudu/127.11.14.126@KRBTEST.COM
I20260807 18:34:06.243500 11321 external_mini_cluster.cc:1428] Setting key cd79865c4bbf5248dc8f364d163a6c48
I20260807 18:34:06.248025 22405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:34:06.235926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.66:41867 (local address 127.11.14.126:35017)
0807 18:34:06.236053 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0807 18:34:06.236057 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:34:06.236915 (+ 858us) server_negotiation.cc:407] Connection header received
0807 18:34:06.238230 (+ 1315us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:34:06.238232 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:34:06.238276 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:34:06.238365 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:34:06.239470 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:34:06.240085 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:34:06.241171 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:34:06.241381 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:34:06.241455 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:34:06.244114 (+ 2659us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:34:06.244121 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:34:06.244123 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:34:06.244148 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:34:06.245634 (+ 1486us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:34:06.246318 (+ 684us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:34:06.246320 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:34:06.246322 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:34:06.246398 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:34:06.246795 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:34:06.246797 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:34:06.246798 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:34:06.246963 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:34:06.247039 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0807 18:34:06.247670 (+ 631us) server_negotiation.cc:299] Negotiation successful
0807 18:34:06.247810 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260807 18:34:06.248795 22706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:34:06.236296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:35017 (local address 127.11.14.66:41867)
0807 18:34:06.236741 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0807 18:34:06.236798 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0807 18:34:06.238040 (+ 1242us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:34:06.238528 (+ 488us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:34:06.238535 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:34:06.238888 (+ 353us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:34:06.239349 (+ 461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:34:06.239360 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:34:06.240197 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:34:06.240199 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:34:06.241069 (+ 870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:34:06.241078 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:34:06.241501 (+ 423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:34:06.241504 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:34:06.241647 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:34:06.242170 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:34:06.242187 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:34:06.243932 (+ 1745us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:34:06.245778 (+ 1846us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:34:06.245786 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:34:06.245804 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:34:06.246209 (+ 405us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:34:06.246489 (+ 280us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:34:06.246496 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:34:06.246500 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:34:06.246701 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:34:06.247065 (+ 364us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:34:06.247072 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:34:06.247405 (+ 333us) client_negotiation.cc:769] Sending connection context
0807 18:34:06.247625 (+ 220us) client_negotiation.cc:240] Negotiation successful
0807 18:34:06.247900 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":112,"threads_started":1}
I20260807 18:34:06.249997 22704 heartbeater.cc:347] Connected to a master server at 127.11.14.126:35017
I20260807 18:34:06.250252 22704 heartbeater.cc:464] Registering TS with master...
I20260807 18:34:06.250734 22704 heartbeater.cc:511] Master 127.11.14.126:35017 requested a full tablet report, sending...
I20260807 18:34:06.251827 22364 ts_manager.cc:194] Registered new tserver with Master: 64bdfa84d7cd495eb6282d2ba8d0cbff (127.11.14.66:36445)
I20260807 18:34:06.252401 22364 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.14.66@KRBTEST.COM'} at 127.11.14.66:41867
WARNING: no policy specified for kudu/127.11.14.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.14.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.14.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.67.keytab.
Entry for principal kudu/127.11.14.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.67.keytab.
I20260807 18:34:06.272274 11321 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkz56XN/build/debug/bin/kudu
/tmp/dist-test-taskkz56XN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127579900523-11321-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127579900523-11321-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127579900523-11321-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127579900523-11321-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.14.67:0
--local_ip_for_outbound_sockets=127.11.14.67
--webserver_interface=127.11.14.67
--webserver_port=0
--tserver_master_addrs=127.11.14.126:35017
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.67.keytab
--principal=kudu/127.11.14.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.14.84:37385
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127579900523-11321-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127579900523-11321-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127579900523-11321-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:34:06.390173 22709 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:34:06.390462 22709 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:34:06.390532 22709 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:34:06.394619 22709 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:34:06.394716 22709 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:34:06.394812 22709 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.14.67
I20260807 18:34:06.400178 22709 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.14.84:37385
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127579900523-11321-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127579900523-11321-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.67.keytab
--principal=kudu/127.11.14.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.14.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127579900523-11321-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.14.67
--webserver_port=0
--tserver_master_addrs=127.11.14.126:35017
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22709
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.14.67
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127579900523-11321-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:34:06.401307 22709 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:34:06.402386 22709 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:34:06.412179 22715 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:34:06.412149 22714 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:34:06.412163 22717 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:34:06.412410 22709 server_base.cc:1034] running on GCE node
I20260807 18:34:06.412859 22709 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:34:06.413528 22709 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:34:06.414687 22709 hybrid_clock.cc:648] HybridClock initialized: now 1786127646414651 us; error 47 us; skew 500 ppm
Aug 07 18:34:06 dist-test-slave-6k22 krb5kdc[22316](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127646, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:34:06.417407 22709 init.cc:377] Logged in from keytab as kudu/127.11.14.67@KRBTEST.COM (short username kudu)
I20260807 18:34:06.418305 22709 webserver.cc:460] Webserver started at http://127.11.14.67:43039/ using document root <none> and password file <none>
I20260807 18:34:06.418884 22709 fs_manager.cc:359] Metadata directory not provided
I20260807 18:34:06.418973 22709 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:34:06.419205 22709 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:34:06.421285 22709 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127579900523-11321-0/minicluster-data/ts-2/data/instance:
uuid: "f9e064917fe54a7a9d3d48dbb9cc6f53"
format_stamp: "Formatted at 2026-08-07 18:34:06 on dist-test-slave-6k22"
server_key: "d3daddce2b379a1d43c687818031b66c"
server_key_iv: "3cf7585422795f68fb90805412d67820"
server_key_version: "encryptionkey@0"
I20260807 18:34:06.421869 22709 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127579900523-11321-0/minicluster-data/ts-2/wal/instance:
uuid: "f9e064917fe54a7a9d3d48dbb9cc6f53"
format_stamp: "Formatted at 2026-08-07 18:34:06 on dist-test-slave-6k22"
server_key: "d3daddce2b379a1d43c687818031b66c"
server_key_iv: "3cf7585422795f68fb90805412d67820"
server_key_version: "encryptionkey@0"
I20260807 18:34:06.425843 22709 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.002s
I20260807 18:34:06.428319 22725 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:34:06.429566 22709 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260807 18:34:06.429698 22709 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127579900523-11321-0/minicluster-data/ts-2/data,/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127579900523-11321-0/minicluster-data/ts-2/wal
uuid: "f9e064917fe54a7a9d3d48dbb9cc6f53"
format_stamp: "Formatted at 2026-08-07 18:34:06 on dist-test-slave-6k22"
server_key: "d3daddce2b379a1d43c687818031b66c"
server_key_iv: "3cf7585422795f68fb90805412d67820"
server_key_version: "encryptionkey@0"
I20260807 18:34:06.429805 22709 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127579900523-11321-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127579900523-11321-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127579900523-11321-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:34:06.444491 22709 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:34:06.447271 22709 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:34:06.447472 22709 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:34:06.448123 22709 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:34:06.448959 22709 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:34:06.449029 22709 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:34:06.449101 22709 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:34:06.449141 22709 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:34:06.457868 22709 rpc_server.cc:307] RPC server started. Bound to: 127.11.14.67:44483
I20260807 18:34:06.457904 22844 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.14.67:44483 every 8 connection(s)
I20260807 18:34:06.458760 22709 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127579900523-11321-0/minicluster-data/ts-2/data/info.pb
Aug 07 18:34:06 dist-test-slave-6k22 krb5kdc[22316](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127646, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.67@KRBTEST.COM for kudu/127.11.14.126@KRBTEST.COM
I20260807 18:34:06.468976 11321 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkz56XN/build/debug/bin/kudu as pid 22709
I20260807 18:34:06.469110 11321 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127579900523-11321-0/minicluster-data/ts-2/wal/instance
I20260807 18:34:06.469422 11321 external_mini_cluster.cc:1428] Setting key f9f0f7e4011db03769ecadabaa1b9c46
I20260807 18:34:06.472432 22405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:34:06.460618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.67:33969 (local address 127.11.14.126:35017)
0807 18:34:06.460746 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0807 18:34:06.460749 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 18:34:06.461613 (+ 864us) server_negotiation.cc:407] Connection header received
0807 18:34:06.462962 (+ 1349us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:34:06.462965 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:34:06.463013 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:34:06.463090 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:34:06.464324 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:34:06.464868 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:34:06.465936 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:34:06.466144 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:34:06.466218 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:34:06.468759 (+ 2541us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:34:06.468763 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:34:06.468765 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:34:06.468799 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:34:06.470399 (+ 1600us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:34:06.470902 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:34:06.470907 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:34:06.470908 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:34:06.470994 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:34:06.471343 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:34:06.471346 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:34:06.471348 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:34:06.471535 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:34:06.471667 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0807 18:34:06.472154 (+ 487us) server_negotiation.cc:299] Negotiation successful
0807 18:34:06.472272 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260807 18:34:06.473234 22847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:34:06.461017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:35017 (local address 127.11.14.67:33969)
0807 18:34:06.461456 (+ 439us) negotiation.cc:107] Waiting for socket to connect
0807 18:34:06.461499 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 18:34:06.462755 (+ 1256us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:34:06.463253 (+ 498us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:34:06.463259 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:34:06.463691 (+ 432us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:34:06.464190 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:34:06.464203 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:34:06.464976 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:34:06.464978 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:34:06.465823 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:34:06.465843 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:34:06.466269 (+ 426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:34:06.466271 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:34:06.466407 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:34:06.466999 (+ 592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:34:06.467015 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:34:06.468586 (+ 1571us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:34:06.470521 (+ 1935us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:34:06.470526 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:34:06.470542 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:34:06.470801 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:34:06.471085 (+ 284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:34:06.471089 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:34:06.471091 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:34:06.471234 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:34:06.471666 (+ 432us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:34:06.471674 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:34:06.471924 (+ 250us) client_negotiation.cc:769] Sending connection context
0807 18:34:06.472112 (+ 188us) client_negotiation.cc:240] Negotiation successful
0807 18:34:06.472444 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":102,"threads_started":1}
I20260807 18:34:06.474272 22845 heartbeater.cc:347] Connected to a master server at 127.11.14.126:35017
I20260807 18:34:06.474478 22845 heartbeater.cc:464] Registering TS with master...
I20260807 18:34:06.474915 22845 heartbeater.cc:511] Master 127.11.14.126:35017 requested a full tablet report, sending...
I20260807 18:34:06.475873 22364 ts_manager.cc:194] Registered new tserver with Master: f9e064917fe54a7a9d3d48dbb9cc6f53 (127.11.14.67:44483)
I20260807 18:34:06.476428 22364 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.14.67@KRBTEST.COM'} at 127.11.14.67:33969
I20260807 18:34:06.483162 11321 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 18:34:06.492328 22405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:34:06.485182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43740 (local address 127.11.14.126:35017)
0807 18:34:06.485305 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0807 18:34:06.485309 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:34:06.485493 (+ 184us) server_negotiation.cc:407] Connection header received
0807 18:34:06.485582 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:34:06.485584 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:34:06.485620 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:34:06.485683 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:34:06.486315 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:34:06.486802 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:34:06.487768 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:34:06.488018 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:34:06.488130 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:34:06.489425 (+ 1295us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:34:06.489429 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:34:06.489432 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:34:06.489456 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:34:06.490594 (+ 1138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:34:06.491016 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:34:06.491021 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:34:06.491023 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:34:06.491109 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:34:06.491384 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:34:06.491389 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:34:06.491391 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:34:06.491678 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:34:06.491814 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0807 18:34:06.492050 (+ 236us) server_negotiation.cc:299] Negotiation successful
0807 18:34:06.492166 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260807 18:34:06.494918 22364 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43740:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 18:34:06.497926 22364 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 18:34:06.512800 22857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:34:06.506781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.66:36445 (local address 127.0.0.1:60292)
0807 18:34:06.507779 (+ 998us) negotiation.cc:107] Waiting for socket to connect
0807 18:34:06.507792 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 18:34:06.507890 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:34:06.508710 (+ 820us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:34:06.508716 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:34:06.508736 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:34:06.508868 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:34:06.508873 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:34:06.509991 (+ 1118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:34:06.509996 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:34:06.511148 (+ 1152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:34:06.511158 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:34:06.512212 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:34:06.512216 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:34:06.512335 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:34:06.512377 (+ 42us) client_negotiation.cc:769] Sending connection context
0807 18:34:06.512493 (+ 116us) client_negotiation.cc:240] Negotiation successful
0807 18:34:06.512611 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":936,"mutex_wait_us":38,"thread_start_us":126,"threads_started":1}
I20260807 18:34:06.512828 22860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:34:06.507525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60292 (local address 127.11.14.66:36445)
0807 18:34:06.508037 (+ 512us) server_negotiation.cc:206] Beginning negotiation
0807 18:34:06.508050 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 18:34:06.508072 (+ 22us) server_negotiation.cc:407] Connection header received
0807 18:34:06.508153 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:34:06.508157 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:34:06.508365 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:34:06.508525 (+ 160us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:34:06.508979 (+ 454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:34:06.509865 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:34:06.511297 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:34:06.512081 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:34:06.512192 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:34:06.512350 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0807 18:34:06.512562 (+ 212us) server_negotiation.cc:299] Negotiation successful
0807 18:34:06.512665 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":411,"thread_start_us":134,"threads_started":1}
I20260807 18:34:06.513473 22859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:34:06.507535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.65:42183 (local address 127.0.0.1:40584)
0807 18:34:06.508646 (+ 1111us) negotiation.cc:107] Waiting for socket to connect
0807 18:34:06.508661 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 18:34:06.508749 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:34:06.509190 (+ 441us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:34:06.509193 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:34:06.509212 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:34:06.509307 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:34:06.509313 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:34:06.510656 (+ 1343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:34:06.510659 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:34:06.511819 (+ 1160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:34:06.511828 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:34:06.513048 (+ 1220us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:34:06.513052 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:34:06.513146 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:34:06.513169 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 18:34:06.513244 (+ 75us) client_negotiation.cc:240] Negotiation successful
0807 18:34:06.513330 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1049,"mutex_wait_us":9,"thread_start_us":107,"threads_started":1}
I20260807 18:34:06.513568 22858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:34:06.506793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39298 (local address 127.11.14.67:44483)
0807 18:34:06.507620 (+ 827us) server_negotiation.cc:206] Beginning negotiation
0807 18:34:06.507635 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 18:34:06.507955 (+ 320us) server_negotiation.cc:407] Connection header received
0807 18:34:06.508042 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:34:06.508048 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:34:06.508234 (+ 186us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:34:06.508413 (+ 179us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:34:06.508879 (+ 466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:34:06.510103 (+ 1224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:34:06.511657 (+ 1554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:34:06.512509 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:34:06.512613 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:34:06.512743 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0807 18:34:06.513318 (+ 575us) server_negotiation.cc:299] Negotiation successful
0807 18:34:06.513424 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":696,"thread_start_us":160,"threads_started":1}
I20260807 18:34:06.513559 22861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:34:06.508099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40584 (local address 127.11.14.65:42183)
0807 18:34:06.508415 (+ 316us) server_negotiation.cc:206] Beginning negotiation
0807 18:34:06.508431 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 18:34:06.508770 (+ 339us) server_negotiation.cc:407] Connection header received
0807 18:34:06.508868 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:34:06.508873 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:34:06.509048 (+ 175us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:34:06.509200 (+ 152us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:34:06.509431 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:34:06.510504 (+ 1073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:34:06.511937 (+ 1433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:34:06.512707 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:34:06.512812 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:34:06.512984 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0807 18:34:06.513308 (+ 324us) server_negotiation.cc:299] Negotiation successful
0807 18:34:06.513411 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":210,"thread_start_us":99,"threads_started":1}
I20260807 18:34:06.513208 22856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:34:06.506681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.67:44483 (local address 127.0.0.1:39298)
0807 18:34:06.507610 (+ 929us) negotiation.cc:107] Waiting for socket to connect
0807 18:34:06.507667 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0807 18:34:06.507803 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:34:06.508597 (+ 794us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:34:06.508602 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:34:06.508621 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:34:06.508738 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:34:06.508748 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:34:06.510256 (+ 1508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:34:06.510260 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:34:06.511490 (+ 1230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:34:06.511501 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:34:06.512810 (+ 1309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:34:06.512814 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:34:06.512930 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:34:06.512959 (+ 29us) client_negotiation.cc:769] Sending connection context
0807 18:34:06.513036 (+ 77us) client_negotiation.cc:240] Negotiation successful
0807 18:34:06.513108 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":172,"mutex_wait_us":2,"thread_start_us":99,"threads_started":1}
I20260807 18:34:06.516361 22495 tablet_service.cc:1467] Processing CreateTablet for tablet d82ece7159f746cb9b12e4232bdbd89e (DEFAULT_TABLE table=test-table [id=84c092ff929c43d4a1253eea4ddc4eab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:34:06.516414 22635 tablet_service.cc:1467] Processing CreateTablet for tablet d82ece7159f746cb9b12e4232bdbd89e (DEFAULT_TABLE table=test-table [id=84c092ff929c43d4a1253eea4ddc4eab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:34:06.516824 22776 tablet_service.cc:1467] Processing CreateTablet for tablet d82ece7159f746cb9b12e4232bdbd89e (DEFAULT_TABLE table=test-table [id=84c092ff929c43d4a1253eea4ddc4eab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:34:06.517447 22495 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d82ece7159f746cb9b12e4232bdbd89e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:34:06.517483 22635 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d82ece7159f746cb9b12e4232bdbd89e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:34:06.517768 22776 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d82ece7159f746cb9b12e4232bdbd89e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:34:06.524115 22862 tablet_bootstrap.cc:492] T d82ece7159f746cb9b12e4232bdbd89e P ce6c17b71ecc4239ad183d7a58e48d13: Bootstrap starting.
I20260807 18:34:06.525527 22863 tablet_bootstrap.cc:492] T d82ece7159f746cb9b12e4232bdbd89e P 64bdfa84d7cd495eb6282d2ba8d0cbff: Bootstrap starting.
I20260807 18:34:06.526678 22862 tablet_bootstrap.cc:654] T d82ece7159f746cb9b12e4232bdbd89e P ce6c17b71ecc4239ad183d7a58e48d13: Neither blocks nor log segments found. Creating new log.
I20260807 18:34:06.527459 22862 log.cc:824] T d82ece7159f746cb9b12e4232bdbd89e P ce6c17b71ecc4239ad183d7a58e48d13: Log is configured to *not* fsync() on all Append() calls
I20260807 18:34:06.527755 22863 tablet_bootstrap.cc:654] T d82ece7159f746cb9b12e4232bdbd89e P 64bdfa84d7cd495eb6282d2ba8d0cbff: Neither blocks nor log segments found. Creating new log.
I20260807 18:34:06.528467 22863 log.cc:824] T d82ece7159f746cb9b12e4232bdbd89e P 64bdfa84d7cd495eb6282d2ba8d0cbff: Log is configured to *not* fsync() on all Append() calls
I20260807 18:34:06.529198 22864 tablet_bootstrap.cc:492] T d82ece7159f746cb9b12e4232bdbd89e P f9e064917fe54a7a9d3d48dbb9cc6f53: Bootstrap starting.
I20260807 18:34:06.529343 22862 tablet_bootstrap.cc:492] T d82ece7159f746cb9b12e4232bdbd89e P ce6c17b71ecc4239ad183d7a58e48d13: No bootstrap required, opened a new log
I20260807 18:34:06.529547 22862 ts_tablet_manager.cc:1397] T d82ece7159f746cb9b12e4232bdbd89e P ce6c17b71ecc4239ad183d7a58e48d13: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260807 18:34:06.530478 22863 tablet_bootstrap.cc:492] T d82ece7159f746cb9b12e4232bdbd89e P 64bdfa84d7cd495eb6282d2ba8d0cbff: No bootstrap required, opened a new log
I20260807 18:34:06.530879 22863 ts_tablet_manager.cc:1397] T d82ece7159f746cb9b12e4232bdbd89e P 64bdfa84d7cd495eb6282d2ba8d0cbff: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260807 18:34:06.531165 22864 tablet_bootstrap.cc:654] T d82ece7159f746cb9b12e4232bdbd89e P f9e064917fe54a7a9d3d48dbb9cc6f53: Neither blocks nor log segments found. Creating new log.
I20260807 18:34:06.532300 22864 log.cc:824] T d82ece7159f746cb9b12e4232bdbd89e P f9e064917fe54a7a9d3d48dbb9cc6f53: Log is configured to *not* fsync() on all Append() calls
I20260807 18:34:06.534031 22864 tablet_bootstrap.cc:492] T d82ece7159f746cb9b12e4232bdbd89e P f9e064917fe54a7a9d3d48dbb9cc6f53: No bootstrap required, opened a new log
I20260807 18:34:06.533998 22862 raft_consensus.cc:348] T d82ece7159f746cb9b12e4232bdbd89e P ce6c17b71ecc4239ad183d7a58e48d13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9e064917fe54a7a9d3d48dbb9cc6f53" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 44483 } } peers { permanent_uuid: "64bdfa84d7cd495eb6282d2ba8d0cbff" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 36445 } } peers { permanent_uuid: "ce6c17b71ecc4239ad183d7a58e48d13" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 42183 } }
I20260807 18:34:06.534238 22864 ts_tablet_manager.cc:1397] T d82ece7159f746cb9b12e4232bdbd89e P f9e064917fe54a7a9d3d48dbb9cc6f53: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.004s
I20260807 18:34:06.534281 22862 raft_consensus.cc:374] T d82ece7159f746cb9b12e4232bdbd89e P ce6c17b71ecc4239ad183d7a58e48d13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:34:06.534359 22862 raft_consensus.cc:729] T d82ece7159f746cb9b12e4232bdbd89e P ce6c17b71ecc4239ad183d7a58e48d13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce6c17b71ecc4239ad183d7a58e48d13, State: Initialized, Role: FOLLOWER
I20260807 18:34:06.534829 22862 consensus_queue.cc:260] T d82ece7159f746cb9b12e4232bdbd89e P ce6c17b71ecc4239ad183d7a58e48d13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9e064917fe54a7a9d3d48dbb9cc6f53" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 44483 } } peers { permanent_uuid: "64bdfa84d7cd495eb6282d2ba8d0cbff" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 36445 } } peers { permanent_uuid: "ce6c17b71ecc4239ad183d7a58e48d13" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 42183 } }
I20260807 18:34:06.534814 22863 raft_consensus.cc:348] T d82ece7159f746cb9b12e4232bdbd89e P 64bdfa84d7cd495eb6282d2ba8d0cbff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9e064917fe54a7a9d3d48dbb9cc6f53" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 44483 } } peers { permanent_uuid: "64bdfa84d7cd495eb6282d2ba8d0cbff" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 36445 } } peers { permanent_uuid: "ce6c17b71ecc4239ad183d7a58e48d13" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 42183 } }
I20260807 18:34:06.535081 22863 raft_consensus.cc:374] T d82ece7159f746cb9b12e4232bdbd89e P 64bdfa84d7cd495eb6282d2ba8d0cbff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:34:06.535163 22863 raft_consensus.cc:729] T d82ece7159f746cb9b12e4232bdbd89e P 64bdfa84d7cd495eb6282d2ba8d0cbff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64bdfa84d7cd495eb6282d2ba8d0cbff, State: Initialized, Role: FOLLOWER
I20260807 18:34:06.535629 22863 consensus_queue.cc:260] T d82ece7159f746cb9b12e4232bdbd89e P 64bdfa84d7cd495eb6282d2ba8d0cbff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9e064917fe54a7a9d3d48dbb9cc6f53" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 44483 } } peers { permanent_uuid: "64bdfa84d7cd495eb6282d2ba8d0cbff" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 36445 } } peers { permanent_uuid: "ce6c17b71ecc4239ad183d7a58e48d13" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 42183 } }
I20260807 18:34:06.535714 22564 heartbeater.cc:503] Master 127.11.14.126:35017 was elected leader, sending a full tablet report...
I20260807 18:34:06.535957 22862 ts_tablet_manager.cc:1428] T d82ece7159f746cb9b12e4232bdbd89e P ce6c17b71ecc4239ad183d7a58e48d13: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260807 18:34:06.536496 22863 ts_tablet_manager.cc:1428] T d82ece7159f746cb9b12e4232bdbd89e P 64bdfa84d7cd495eb6282d2ba8d0cbff: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260807 18:34:06.536803 22704 heartbeater.cc:503] Master 127.11.14.126:35017 was elected leader, sending a full tablet report...
I20260807 18:34:06.537773 22864 raft_consensus.cc:348] T d82ece7159f746cb9b12e4232bdbd89e P f9e064917fe54a7a9d3d48dbb9cc6f53 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9e064917fe54a7a9d3d48dbb9cc6f53" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 44483 } } peers { permanent_uuid: "64bdfa84d7cd495eb6282d2ba8d0cbff" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 36445 } } peers { permanent_uuid: "ce6c17b71ecc4239ad183d7a58e48d13" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 42183 } }
I20260807 18:34:06.538044 22864 raft_consensus.cc:374] T d82ece7159f746cb9b12e4232bdbd89e P f9e064917fe54a7a9d3d48dbb9cc6f53 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:34:06.538098 22864 raft_consensus.cc:729] T d82ece7159f746cb9b12e4232bdbd89e P f9e064917fe54a7a9d3d48dbb9cc6f53 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9e064917fe54a7a9d3d48dbb9cc6f53, State: Initialized, Role: FOLLOWER
I20260807 18:34:06.538544 22864 consensus_queue.cc:260] T d82ece7159f746cb9b12e4232bdbd89e P f9e064917fe54a7a9d3d48dbb9cc6f53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9e064917fe54a7a9d3d48dbb9cc6f53" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 44483 } } peers { permanent_uuid: "64bdfa84d7cd495eb6282d2ba8d0cbff" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 36445 } } peers { permanent_uuid: "ce6c17b71ecc4239ad183d7a58e48d13" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 42183 } }
I20260807 18:34:06.539204 22845 heartbeater.cc:503] Master 127.11.14.126:35017 was elected leader, sending a full tablet report...
I20260807 18:34:06.539505 22864 ts_tablet_manager.cc:1428] T d82ece7159f746cb9b12e4232bdbd89e P f9e064917fe54a7a9d3d48dbb9cc6f53: Time spent starting tablet: real 0.005s user 0.005s sys 0.001s
W20260807 18:34:06.709748 22846 tablet.cc:2369] T d82ece7159f746cb9b12e4232bdbd89e P f9e064917fe54a7a9d3d48dbb9cc6f53: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 18:34:06.735226 22705 tablet.cc:2369] T d82ece7159f746cb9b12e4232bdbd89e P 64bdfa84d7cd495eb6282d2ba8d0cbff: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 18:34:06.756793 22565 tablet.cc:2369] T d82ece7159f746cb9b12e4232bdbd89e P ce6c17b71ecc4239ad183d7a58e48d13: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:34:06.788738 22868 raft_consensus.cc:482] T d82ece7159f746cb9b12e4232bdbd89e P ce6c17b71ecc4239ad183d7a58e48d13 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:34:06.789002 22868 raft_consensus.cc:504] T d82ece7159f746cb9b12e4232bdbd89e P ce6c17b71ecc4239ad183d7a58e48d13 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9e064917fe54a7a9d3d48dbb9cc6f53" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 44483 } } peers { permanent_uuid: "64bdfa84d7cd495eb6282d2ba8d0cbff" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 36445 } } peers { permanent_uuid: "ce6c17b71ecc4239ad183d7a58e48d13" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 42183 } }
I20260807 18:34:06.790238 22868 leader_election.cc:288] T d82ece7159f746cb9b12e4232bdbd89e P ce6c17b71ecc4239ad183d7a58e48d13 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f9e064917fe54a7a9d3d48dbb9cc6f53 (127.11.14.67:44483), 64bdfa84d7cd495eb6282d2ba8d0cbff (127.11.14.66:36445)
I20260807 18:34:06.794363 22860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:34:06.790625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.65:40347 (local address 127.11.14.66:36445)
0807 18:34:06.790793 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0807 18:34:06.790798 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:34:06.791057 (+ 259us) server_negotiation.cc:407] Connection header received
0807 18:34:06.791217 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:34:06.791220 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:34:06.791272 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:34:06.791333 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:34:06.791752 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:34:06.792264 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:34:06.793371 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:34:06.793767 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:34:06.793903 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:34:06.793957 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0807 18:34:06.794150 (+ 193us) server_negotiation.cc:299] Negotiation successful
0807 18:34:06.794230 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260807 18:34:06.794379 22872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:34:06.790502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.66:36445 (local address 127.11.14.65:40347)
0807 18:34:06.790941 (+ 439us) negotiation.cc:107] Waiting for socket to connect
0807 18:34:06.790953 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 18:34:06.791109 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:34:06.791374 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:34:06.791377 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:34:06.791393 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:34:06.791638 (+ 245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:34:06.791646 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:34:06.792374 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:34:06.792379 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:34:06.793243 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:34:06.793253 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:34:06.793900 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:34:06.793904 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:34:06.794007 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:34:06.794044 (+ 37us) client_negotiation.cc:769] Sending connection context
0807 18:34:06.794112 (+ 68us) client_negotiation.cc:240] Negotiation successful
0807 18:34:06.794195 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":369,"mutex_wait_us":187,"spinlock_wait_cycles":9728,"thread_start_us":99,"threads_started":1}
I20260807 18:34:06.794416 22858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:34:06.790587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.65:35691 (local address 127.11.14.67:44483)
0807 18:34:06.790761 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0807 18:34:06.790767 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:34:06.791033 (+ 266us) server_negotiation.cc:407] Connection header received
0807 18:34:06.791235 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:34:06.791237 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:34:06.791288 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:34:06.791379 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:34:06.791789 (+ 410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:34:06.792263 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:34:06.793404 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:34:06.793788 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:34:06.793924 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:34:06.793979 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0807 18:34:06.794219 (+ 240us) server_negotiation.cc:299] Negotiation successful
0807 18:34:06.794303 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260807 18:34:06.794600 22871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:34:06.790494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.67:44483 (local address 127.11.14.65:35691)
0807 18:34:06.790929 (+ 435us) negotiation.cc:107] Waiting for socket to connect
0807 18:34:06.790941 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 18:34:06.791108 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:34:06.791411 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:34:06.791414 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:34:06.791439 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:34:06.791649 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:34:06.791657 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:34:06.792405 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:34:06.792408 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:34:06.793253 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:34:06.793262 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:34:06.793929 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:34:06.793932 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:34:06.794072 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:34:06.794089 (+ 17us) client_negotiation.cc:769] Sending connection context
0807 18:34:06.794184 (+ 95us) client_negotiation.cc:240] Negotiation successful
0807 18:34:06.794248 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":361,"mutex_wait_us":76,"spinlock_wait_cycles":46336,"thread_start_us":156,"threads_started":1}
I20260807 18:34:06.795050 22656 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d82ece7159f746cb9b12e4232bdbd89e" candidate_uuid: "ce6c17b71ecc4239ad183d7a58e48d13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64bdfa84d7cd495eb6282d2ba8d0cbff" is_pre_election: true
I20260807 18:34:06.795289 22797 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d82ece7159f746cb9b12e4232bdbd89e" candidate_uuid: "ce6c17b71ecc4239ad183d7a58e48d13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9e064917fe54a7a9d3d48dbb9cc6f53" is_pre_election: true
I20260807 18:34:06.795362 22656 raft_consensus.cc:2454] T d82ece7159f746cb9b12e4232bdbd89e P 64bdfa84d7cd495eb6282d2ba8d0cbff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce6c17b71ecc4239ad183d7a58e48d13 in term 0.
I20260807 18:34:06.795621 22797 raft_consensus.cc:2454] T d82ece7159f746cb9b12e4232bdbd89e P f9e064917fe54a7a9d3d48dbb9cc6f53 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce6c17b71ecc4239ad183d7a58e48d13 in term 0.
I20260807 18:34:06.795876 22446 leader_election.cc:302] T d82ece7159f746cb9b12e4232bdbd89e P ce6c17b71ecc4239ad183d7a58e48d13 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64bdfa84d7cd495eb6282d2ba8d0cbff, ce6c17b71ecc4239ad183d7a58e48d13; no voters:
I20260807 18:34:06.796162 22868 raft_consensus.cc:2781] T d82ece7159f746cb9b12e4232bdbd89e P ce6c17b71ecc4239ad183d7a58e48d13 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:34:06.796240 22868 raft_consensus.cc:482] T d82ece7159f746cb9b12e4232bdbd89e P ce6c17b71ecc4239ad183d7a58e48d13 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:34:06.796293 22868 raft_consensus.cc:3037] T d82ece7159f746cb9b12e4232bdbd89e P ce6c17b71ecc4239ad183d7a58e48d13 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:34:06.797665 22868 raft_consensus.cc:504] T d82ece7159f746cb9b12e4232bdbd89e P ce6c17b71ecc4239ad183d7a58e48d13 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9e064917fe54a7a9d3d48dbb9cc6f53" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 44483 } } peers { permanent_uuid: "64bdfa84d7cd495eb6282d2ba8d0cbff" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 36445 } } peers { permanent_uuid: "ce6c17b71ecc4239ad183d7a58e48d13" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 42183 } }
I20260807 18:34:06.798048 22868 leader_election.cc:288] T d82ece7159f746cb9b12e4232bdbd89e P ce6c17b71ecc4239ad183d7a58e48d13 [CANDIDATE]: Term 1 election: Requested vote from peers f9e064917fe54a7a9d3d48dbb9cc6f53 (127.11.14.67:44483), 64bdfa84d7cd495eb6282d2ba8d0cbff (127.11.14.66:36445)
I20260807 18:34:06.798432 22797 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d82ece7159f746cb9b12e4232bdbd89e" candidate_uuid: "ce6c17b71ecc4239ad183d7a58e48d13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9e064917fe54a7a9d3d48dbb9cc6f53"
I20260807 18:34:06.798470 22656 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d82ece7159f746cb9b12e4232bdbd89e" candidate_uuid: "ce6c17b71ecc4239ad183d7a58e48d13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64bdfa84d7cd495eb6282d2ba8d0cbff"
I20260807 18:34:06.798583 22797 raft_consensus.cc:3037] T d82ece7159f746cb9b12e4232bdbd89e P f9e064917fe54a7a9d3d48dbb9cc6f53 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:34:06.798583 22656 raft_consensus.cc:3037] T d82ece7159f746cb9b12e4232bdbd89e P 64bdfa84d7cd495eb6282d2ba8d0cbff [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:34:06.799959 22656 raft_consensus.cc:2454] T d82ece7159f746cb9b12e4232bdbd89e P 64bdfa84d7cd495eb6282d2ba8d0cbff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce6c17b71ecc4239ad183d7a58e48d13 in term 1.
I20260807 18:34:06.799959 22797 raft_consensus.cc:2454] T d82ece7159f746cb9b12e4232bdbd89e P f9e064917fe54a7a9d3d48dbb9cc6f53 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce6c17b71ecc4239ad183d7a58e48d13 in term 1.
I20260807 18:34:06.800271 22447 leader_election.cc:302] T d82ece7159f746cb9b12e4232bdbd89e P ce6c17b71ecc4239ad183d7a58e48d13 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce6c17b71ecc4239ad183d7a58e48d13, f9e064917fe54a7a9d3d48dbb9cc6f53; no voters:
I20260807 18:34:06.800451 22868 raft_consensus.cc:2781] T d82ece7159f746cb9b12e4232bdbd89e P ce6c17b71ecc4239ad183d7a58e48d13 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:34:06.800707 22868 raft_consensus.cc:686] T d82ece7159f746cb9b12e4232bdbd89e P ce6c17b71ecc4239ad183d7a58e48d13 [term 1 LEADER]: Becoming Leader. State: Replica: ce6c17b71ecc4239ad183d7a58e48d13, State: Running, Role: LEADER
I20260807 18:34:06.801054 22868 consensus_queue.cc:237] T d82ece7159f746cb9b12e4232bdbd89e P ce6c17b71ecc4239ad183d7a58e48d13 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9e064917fe54a7a9d3d48dbb9cc6f53" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 44483 } } peers { permanent_uuid: "64bdfa84d7cd495eb6282d2ba8d0cbff" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 36445 } } peers { permanent_uuid: "ce6c17b71ecc4239ad183d7a58e48d13" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 42183 } }
I20260807 18:34:06.804239 22355 catalog_manager.cc:5495] T d82ece7159f746cb9b12e4232bdbd89e P ce6c17b71ecc4239ad183d7a58e48d13 reported cstate change: term changed from 0 to 1, leader changed from <none> to ce6c17b71ecc4239ad183d7a58e48d13 (127.11.14.65). New cstate: current_term: 1 leader_uuid: "ce6c17b71ecc4239ad183d7a58e48d13" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9e064917fe54a7a9d3d48dbb9cc6f53" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 44483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64bdfa84d7cd495eb6282d2ba8d0cbff" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 36445 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce6c17b71ecc4239ad183d7a58e48d13" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 42183 } health_report { overall_health: HEALTHY } } }
I20260807 18:34:06.876756 22861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:34:06.873416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40596 (local address 127.11.14.65:42183)
0807 18:34:06.873556 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0807 18:34:06.873560 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:34:06.873575 (+ 15us) server_negotiation.cc:407] Connection header received
0807 18:34:06.873681 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:34:06.873683 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:34:06.873738 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:34:06.873837 (+ 99us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:34:06.874068 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:34:06.874557 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:34:06.875365 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:34:06.875603 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:34:06.875679 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:34:06.875968 (+ 289us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:34:06.876385 (+ 417us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:34:06.876477 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0807 18:34:06.876536 (+ 59us) server_negotiation.cc:299] Negotiation successful
0807 18:34:06.876582 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260807 18:34:06.882649 22797 raft_consensus.cc:1261] T d82ece7159f746cb9b12e4232bdbd89e P f9e064917fe54a7a9d3d48dbb9cc6f53 [term 1 FOLLOWER]: Refusing update from remote peer ce6c17b71ecc4239ad183d7a58e48d13: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:34:06.882649 22656 raft_consensus.cc:1261] T d82ece7159f746cb9b12e4232bdbd89e P 64bdfa84d7cd495eb6282d2ba8d0cbff [term 1 FOLLOWER]: Refusing update from remote peer ce6c17b71ecc4239ad183d7a58e48d13: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:34:06.883381 22868 consensus_queue.cc:1035] T d82ece7159f746cb9b12e4232bdbd89e P ce6c17b71ecc4239ad183d7a58e48d13 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f9e064917fe54a7a9d3d48dbb9cc6f53" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 44483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:34:06.883632 22873 consensus_queue.cc:1035] T d82ece7159f746cb9b12e4232bdbd89e P ce6c17b71ecc4239ad183d7a58e48d13 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64bdfa84d7cd495eb6282d2ba8d0cbff" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 36445 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:34:06.892352 22879 mvcc.cc:205] Tried to move back new op lower bound from 7315978841624117248 to 7315978841299607552. Current Snapshot: MvccSnapshot[applied={T|T < 7315978841624117248}]
I20260807 18:34:06.900862 22355 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43740:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:34:06.901041 22355 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43740:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:34:06.903789 22355 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 087469ff056948aba87aecb0a72c76d2: Sending DeleteTablet for 3 replicas of tablet d82ece7159f746cb9b12e4232bdbd89e
I20260807 18:34:06.904564 22635 tablet_service.cc:1514] Processing DeleteTablet for tablet d82ece7159f746cb9b12e4232bdbd89e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:34:06 UTC) from {username='kudu', principal='kudu/127.11.14.126@KRBTEST.COM'} at 127.0.0.1:60292
I20260807 18:34:06.904740 22776 tablet_service.cc:1514] Processing DeleteTablet for tablet d82ece7159f746cb9b12e4232bdbd89e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:34:06 UTC) from {username='kudu', principal='kudu/127.11.14.126@KRBTEST.COM'} at 127.0.0.1:39298
I20260807 18:34:06.904739 22495 tablet_service.cc:1514] Processing DeleteTablet for tablet d82ece7159f746cb9b12e4232bdbd89e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:34:06 UTC) from {username='kudu', principal='kudu/127.11.14.126@KRBTEST.COM'} at 127.0.0.1:40584
I20260807 18:34:06.905275 22888 tablet_replica.cc:330] T d82ece7159f746cb9b12e4232bdbd89e P 64bdfa84d7cd495eb6282d2ba8d0cbff: stopping tablet replica
I20260807 18:34:06.905648 11321 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkz56XN/build/debug/bin/kudu with pid 22428
I20260807 18:34:06.905678 22888 raft_consensus.cc:2229] T d82ece7159f746cb9b12e4232bdbd89e P 64bdfa84d7cd495eb6282d2ba8d0cbff [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:34:06.910957 22889 tablet_replica.cc:330] T d82ece7159f746cb9b12e4232bdbd89e P f9e064917fe54a7a9d3d48dbb9cc6f53: stopping tablet replica
I20260807 18:34:06.911020 22888 raft_consensus.cc:2258] T d82ece7159f746cb9b12e4232bdbd89e P 64bdfa84d7cd495eb6282d2ba8d0cbff [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:34:06.911312 22889 raft_consensus.cc:2229] T d82ece7159f746cb9b12e4232bdbd89e P f9e064917fe54a7a9d3d48dbb9cc6f53 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:34:06.911711 22889 raft_consensus.cc:2258] T d82ece7159f746cb9b12e4232bdbd89e P f9e064917fe54a7a9d3d48dbb9cc6f53 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:34:06.912983 22889 ts_tablet_manager.cc:1905] T d82ece7159f746cb9b12e4232bdbd89e P f9e064917fe54a7a9d3d48dbb9cc6f53: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:34:06.912983 22888 ts_tablet_manager.cc:1905] T d82ece7159f746cb9b12e4232bdbd89e P 64bdfa84d7cd495eb6282d2ba8d0cbff: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:34:06.915879 22888 ts_tablet_manager.cc:1918] T d82ece7159f746cb9b12e4232bdbd89e P 64bdfa84d7cd495eb6282d2ba8d0cbff: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:34:06.915879 22889 ts_tablet_manager.cc:1918] T d82ece7159f746cb9b12e4232bdbd89e P f9e064917fe54a7a9d3d48dbb9cc6f53: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:34:06.915987 22888 log.cc:1196] T d82ece7159f746cb9b12e4232bdbd89e P 64bdfa84d7cd495eb6282d2ba8d0cbff: Deleting WAL directory at /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127579900523-11321-0/minicluster-data/ts-1/wal/wals/d82ece7159f746cb9b12e4232bdbd89e
I20260807 18:34:06.915987 22889 log.cc:1196] T d82ece7159f746cb9b12e4232bdbd89e P f9e064917fe54a7a9d3d48dbb9cc6f53: Deleting WAL directory at /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786127579900523-11321-0/minicluster-data/ts-2/wal/wals/d82ece7159f746cb9b12e4232bdbd89e
I20260807 18:34:06.916342 22888 ts_tablet_manager.cc:1939] T d82ece7159f746cb9b12e4232bdbd89e P 64bdfa84d7cd495eb6282d2ba8d0cbff: Deleting consensus metadata
I20260807 18:34:06.916342 22889 ts_tablet_manager.cc:1939] T d82ece7159f746cb9b12e4232bdbd89e P f9e064917fe54a7a9d3d48dbb9cc6f53: Deleting consensus metadata
I20260807 18:34:06.917657 22348 catalog_manager.cc:4841] TS f9e064917fe54a7a9d3d48dbb9cc6f53 (127.11.14.67:44483): tablet d82ece7159f746cb9b12e4232bdbd89e (table test-table [id=84c092ff929c43d4a1253eea4ddc4eab]) successfully deleted
I20260807 18:34:06.917657 22347 catalog_manager.cc:4841] TS 64bdfa84d7cd495eb6282d2ba8d0cbff (127.11.14.66:36445): tablet d82ece7159f746cb9b12e4232bdbd89e (table test-table [id=84c092ff929c43d4a1253eea4ddc4eab]) successfully deleted
W20260807 18:34:06.918481 22346 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.14.65:42183 (error 108)
I20260807 18:34:06.919303 11321 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkz56XN/build/debug/bin/kudu with pid 22569
I20260807 18:34:06.919600 22856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:34:06.919244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.65:42183 (local address 127.0.0.1:40612)
0807 18:34:06.919385 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0807 18:34:06.919480 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.65:42183: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260807 18:34:06.923542 22346 catalog_manager.cc:4584] TS ce6c17b71ecc4239ad183d7a58e48d13 (127.11.14.65:42183): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d82ece7159f746cb9b12e4232bdbd89e: Network error: Client connection negotiation failed: client connection to 127.11.14.65:42183: connect: Connection refused (error 111)
I20260807 18:34:06.933853 11321 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkz56XN/build/debug/bin/kudu with pid 22709
I20260807 18:34:06.941653 11321 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkz56XN/build/debug/bin/kudu with pid 22329
2026-08-07T18:34:06Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2681 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127579900523-11321-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:34:07 dist-test-slave-6k22 krb5kdc[22896](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:34:07 dist-test-slave-6k22 krb5kdc[22896](info): set up 2 sockets
Aug 07 18:34:07 dist-test-slave-6k22 krb5kdc[22896](info): commencing operation
krb5kdc: starting...
W20260807 18:34:08.043048 11321 mini_kdc.cc:121] Time spent starting KDC: real 1.077s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127579900523-11321-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127579900523-11321-0/krb5kdc/test-user.keytab.
Aug 07 18:34:08 dist-test-slave-6k22 krb5kdc[22896](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127648, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:34:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:34:08Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.11.14.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.11.14.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.11.14.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127579900523-11321-0/krb5kdc/rangeradmin_127.11.14.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.11.14.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127579900523-11321-0/krb5kdc/rangeradmin_127.11.14.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.11.14.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.11.14.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.11.14.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127579900523-11321-0/krb5kdc/rangerlookup_127.11.14.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.11.14.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127579900523-11321-0/krb5kdc/rangerlookup_127.11.14.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.11.14.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.11.14.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.11.14.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127579900523-11321-0/krb5kdc/HTTP_127.11.14.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.11.14.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127579900523-11321-0/krb5kdc/HTTP_127.11.14.84@KRBTEST.COM.keytab.
I20260807 18:34:08.220410 11321 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127579900523-11321-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskkz56XN/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127579900523-11321-0/minicluster-data/postgres -l logfile start
2026-08-07 18:34:10.767 UTC [22927] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-07 18:34:10.767 UTC [22927] LOG: listening on IPv4 address "127.11.14.84", port 53471
2026-08-07 18:34:10.772 UTC [22927] LOG: listening on Unix socket "/tmp/.s.PGSQL.53471"
2026-08-07 18:34:10.794 UTC [22930] LOG: database system was shut down at 2026-08-07 18:34:09 UTC
2026-08-07 18:34:10.797 UTC [22927] LOG: database system is ready to accept connections
I20260807 18:34:11.803740 11321 mini_postgres.cc:96] Postgres bound to 53471
2026-08-07 18:34:11.811 UTC [22938] FATAL: database "slave" does not exist
127.11.14.84:53471 - accepting connections
I20260807 18:34:11.811817 11321 mini_ranger.cc:161] Starting Ranger...
I20260807 18:34:11.827342 11321 mini_ranger.cc:84] Created miniranger Postgres user
I20260807 18:34:12.306205 11321 mini_ranger.cc:87] Created ranger Postgres database
I20260807 18:34:12.306291 11321 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127579900523-11321-0/minicluster-data/ranger-admin
2026-08-07 18:34:12,600 [I] DB FLAVOR :POSTGRES
2026-08-07 18:34:12,600 [I] --------- Verifying Ranger DB connection ---------
2026-08-07 18:34:12,600 [I] Checking connection..
2026-08-07 18:34:12,600 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskkz56XN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskkz56XN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.14.84:53471/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-07 18:34:12,844 [I] Checking connection passed.
2026-08-07 18:34:12,844 [I] --------- Verifying version history table ---------
2026-08-07 18:34:12,845 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskkz56XN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskkz56XN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.14.84:53471/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 18:34:13,097 [I] Table x_db_version_h does not exist in database ranger
2026-08-07 18:34:13,097 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskkz56XN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskkz56XN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.14.84:53471/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 18:34:13,374 [I] Table x_db_version_h does not exist in database ranger
2026-08-07 18:34:13,374 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-07 18:34:13,374 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskkz56XN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskkz56XN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.14.84:53471/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskkz56XN/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-07 18:34:13.636 UTC [23008] WARNING: there is no transaction in progress
2026-08-07 18:34:13,643 [I] create_dbversion_catalog.sql file imported successfully
2026-08-07 18:34:13,644 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskkz56XN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskkz56XN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.14.84:53471/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 18:34:13,915 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-07 18:34:13,915 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-07 18:34:13,915 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskkz56XN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskkz56XN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.14.84:53471/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-07 18:34:14,166 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskkz56XN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskkz56XN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.14.84:53471/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-07 18:34:14,442 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskkz56XN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskkz56XN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.14.84:53471/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-6k22.c.gcp-upstream.internal','N') ;"
2026-08-07 18:34:14,713 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskkz56XN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskkz56XN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.14.84:53471/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-07 18:34:14,989 [I] Table x_portal_user does not exist in database ranger
2026-08-07 18:34:14,990 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskkz56XN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskkz56XN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.14.84:53471/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-07 18:34:15,257 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-07 18:34:15,257 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskkz56XN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskkz56XN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.14.84:53471/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-07 18:34:15,522 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskkz56XN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskkz56XN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.14.84:53471/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-07 18:34:15,773 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-07 18:34:15,774 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskkz56XN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskkz56XN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.14.84:53471/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskkz56XN/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-07 18:34:16.758 UTC [23126] WARNING: there is no transaction in progress
2026-08-07 18:34:16.776 UTC [23126] WARNING: there is no transaction in progress
2026-08-07 18:34:16.794 UTC [23126] WARNING: there is no transaction in progress
2026-08-07 18:34:16.811 UTC [23126] WARNING: there is no transaction in progress
2026-08-07 18:34:16.830 UTC [23126] WARNING: there is no transaction in progress
2026-08-07 18:34:16.850 UTC [23126] WARNING: there is no transaction in progress
2026-08-07 18:34:16.905 UTC [23126] WARNING: there is no transaction in progress
2026-08-07 18:34:16.912 UTC [23126] WARNING: there is no transaction in progress
2026-08-07 18:34:16.927 UTC [23126] WARNING: there is no transaction in progress
2026-08-07 18:34:16.937 UTC [23126] WARNING: there is no transaction in progress
2026-08-07 18:34:16.950 UTC [23126] WARNING: there is no transaction in progress
2026-08-07 18:34:16.964 UTC [23126] WARNING: there is no transaction in progress
2026-08-07 18:34:16.974 UTC [23126] WARNING: there is no transaction in progress
2026-08-07 18:34:16.981 UTC [23126] WARNING: there is no transaction in progress
2026-08-07 18:34:16.988 UTC [23126] WARNING: there is no transaction in progress
2026-08-07 18:34:16.994 UTC [23126] WARNING: there is no transaction in progress
2026-08-07 18:34:17,626 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-07 18:34:17,626 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskkz56XN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskkz56XN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.14.84:53471/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-07 18:34:17,887 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-07 18:34:17,887 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskkz56XN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskkz56XN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.14.84:53471/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-07 18:34:18,180 [I] Table x_portal_user already exists in database 'ranger'
2026-08-07 18:34:18,181 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskkz56XN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskkz56XN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.14.84:53471/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-07 18:34:18,451 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-07 18:34:18,452 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskkz56XN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskkz56XN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.14.84:53471/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-07 18:34:18,728 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskkz56XN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskkz56XN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.14.84:53471/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-07 18:34:19,004 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskkz56XN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskkz56XN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.14.84:53471/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-6k22.c.gcp-upstream.internal';"
2026-08-07 18:34:19,268 [I] CORE_DB_SCHEMA import status has been updated
2026-08-07 18:34:19,269 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskkz56XN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskkz56XN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.14.84:53471/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-07 18:34:19,534 [I] DB_PATCHES have already been applied
I20260807 18:34:19.540827 11321 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127579900523-11321-0/minicluster-data/ranger-admin:/tmp/dist-test-taskkz56XN/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskkz56XN/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskkz56XN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskkz56XN/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260807 18:34:19.540930 11321 mini_ranger.cc:193] Using host: 127.11.14.84
I20260807 18:34:19.545601 11321 mini_ranger.cc:227] Ranger admin URL: http://127.11.14.84:58391
Aug 07, 2026 6:34:19 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 07, 2026 6:34:19 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 07, 2026 6:34:19 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127579900523-11321-0/minicluster-data/ranger-admin/ews/webapp
Aug 07, 2026 6:34:19 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127579900523-11321-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 07, 2026 6:34:19 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127579900523-11321-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 07, 2026 6:34:20 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127579900523-11321-0/minicluster-data/ranger-admin/ews/webapp].
Aug 07, 2026 6:34:20 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260807 18:34:20.752099 11321 mini_ranger.cc:160] Time spent starting Ranger: real 8.940s user 0.005s sys 0.008s
Aug 07, 2026 6:34:42 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 07, 2026 6:34:42 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 07, 2026 6:34:42 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 07, 2026 6:34:42 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 07, 2026 6:34:42 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 07, 2026 6:34:42 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 07, 2026 6:34:42 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 07, 2026 6:34:42 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 07, 2026 6:34:42 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 07, 2026 6:34:42 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 07, 2026 6:34:42 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 07, 2026 6:34:42 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 07, 2026 6:34:42 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 07, 2026 6:34:42 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 07, 2026 6:34:42 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 07, 2026 6:34:42 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 07, 2026 6:34:42 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 07, 2026 6:34:43 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260807 18:34:43.795419 11321 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.11.14.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.11.14.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.11.14.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127579900523-11321-0/krb5kdc/rangerkms_127.11.14.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.11.14.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127579900523-11321-0/krb5kdc/rangerkms_127.11.14.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.11.14.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.11.14.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.11.14.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127579900523-11321-0/krb5kdc/HTTP_127.11.14.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.11.14.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127579900523-11321-0/krb5kdc/HTTP_127.11.14.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 07 18:34:43 dist-test-slave-6k22 krb5kdc[22896](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127683, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260807 18:34:43.888996 11321 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260807 18:34:43.900554 11321 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260807 18:34:44.768096 11321 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260807 18:34:44.768195 11321 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127579900523-11321-0/minicluster-data/ranger-kms
I20260807 18:34:44.768307 11321 mini_ranger_kms.cc:227] Using postgres at 127.11.14.84:53471
2026-08-07 18:34:45,156 [I] DB FLAVOR :POSTGRES
2026-08-07 18:34:45,157 [I] --------- Verifying Ranger DB connection ---------
2026-08-07 18:34:45,157 [I] Checking connection
2026-08-07 18:34:45,157 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskkz56XN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127579900523-11321-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.14.84:53471/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-07 18:34:45,420 [I] connection success
2026-08-07 18:34:45,421 [I] --------- Verifying Ranger DB tables ---------
2026-08-07 18:34:45,421 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-07 18:34:45,421 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskkz56XN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127579900523-11321-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.14.84:53471/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-07 18:34:45,688 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-07 18:34:45,688 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-07 18:34:45,688 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-07 18:34:45,689 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskkz56XN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127579900523-11321-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.14.84:53471/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127579900523-11321-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-07 18:34:45,958 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260807 18:34:46.467908 11321 mini_ranger_kms.cc:310] Created kms service
I20260807 18:34:46.676954 11321 mini_ranger_kms.cc:326] Created kudu user
I20260807 18:34:46.723982 11321 mini_ranger_kms.cc:343] Created rangerkms user
Aug 07 18:34:46 dist-test-slave-6k22 krb5kdc[22896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127683, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.11.14.84@KRBTEST.COM
I20260807 18:34:47.152506 11321 mini_ranger_kms.cc:384] Added ranger policy
I20260807 18:34:47.152694 11321 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127579900523-11321-0/minicluster-data/ranger-kms:/tmp/dist-test-taskkz56XN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskkz56XN/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskkz56XN/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260807 18:34:47.152725 11321 mini_ranger_kms.cc:240] Using host: 127.11.14.84
I20260807 18:34:47.158609 11321 mini_ranger_kms.cc:276] Ranger KMS PID: 23321
I20260807 18:34:47.158835 11321 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.11.14.84:59359
Aug 07, 2026 6:34:48 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 07, 2026 6:34:48 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 07, 2026 6:34:48 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127579900523-11321-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 07, 2026 6:34:48 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127579900523-11321-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 07, 2026 6:34:49 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127579900523-11321-0/minicluster-data/ranger-kms/ews/webapp].
Aug 07, 2026 6:34:49 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260807 18:34:49.737159 11321 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.848s user 0.011s sys 0.361s
I20260807 18:34:49.737318 11321 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260807 18:34:49.737433 11321 mini_ranger_kms.cc:401] 127.11.14.84:59359/kms/v1/keys
Aug 07 18:34:49 dist-test-slave-6k22 krb5kdc[22896](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786127689, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.11.14.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 07 18:34:50 dist-test-slave-6k22 krb5kdc[22896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127689, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.11.14.84@KRBTEST.COM for HTTP/127.11.14.84@KRBTEST.COM
Aug 07 18:34:50 dist-test-slave-6k22 krb5kdc[22896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127689, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.11.14.84@KRBTEST.COM for HTTP/127.11.14.84@KRBTEST.COM
Aug 07 18:34:53 dist-test-slave-6k22 krb5kdc[22896](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127693, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.11.14.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.14.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.14.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.126.keytab.
Entry for principal kudu/127.11.14.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.126.keytab.
I20260807 18:34:53.130749 11321 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkz56XN/build/debug/bin/kudu
/tmp/dist-test-taskkz56XN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127579900523-11321-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127579900523-11321-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127579900523-11321-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127579900523-11321-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.14.126:45731
--webserver_interface=127.11.14.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.126.keytab
--principal=kudu/127.11.14.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.14.84:38715
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.14.126:45731
--ranger_config_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127579900523-11321-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.14.84:59359/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127579900523-11321-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127579900523-11321-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127579900523-11321-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:34:53.268595 23373 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:34:53.268884 23373 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:34:53.268929 23373 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:34:53.273269 23373 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:34:53.273326 23373 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:34:53.273346 23373 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:34:53.273363 23373 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:34:53.273378 23373 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:34:53.278904 23373 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.14.84:38715
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127579900523-11321-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127579900523-11321-0/minicluster-data/master-0/wal
--ranger_kms_url=127.11.14.84:59359/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.11.14.126:45731
--ranger_config_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127579900523-11321-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.126.keytab
--principal=kudu/127.11.14.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.14.126:45731
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127579900523-11321-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.14.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23373
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127579900523-11321-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:34:53.279963 23373 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:34:53.280941 23373 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:34:53.290637 23381 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:34:53.290680 23380 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:34:53.290702 23373 server_base.cc:1034] running on GCE node
W20260807 18:34:53.290652 23383 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:34:53.291420 23373 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:34:53.292670 23373 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:34:53.293848 23373 hybrid_clock.cc:648] HybridClock initialized: now 1786127693293819 us; error 38 us; skew 500 ppm
Aug 07 18:34:53 dist-test-slave-6k22 krb5kdc[22896](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127693, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:34:53.303933 23373 init.cc:377] Logged in from keytab as kudu/127.11.14.126@KRBTEST.COM (short username kudu)
I20260807 18:34:53.305081 23373 webserver.cc:460] Webserver started at http://127.11.14.126:34969/ using document root <none> and password file <none>
I20260807 18:34:53.305773 23373 fs_manager.cc:359] Metadata directory not provided
I20260807 18:34:53.305873 23373 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:34:53.306166 23373 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 18:34:53 dist-test-slave-6k22 krb5kdc[22896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127693, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.126@KRBTEST.COM for HTTP/127.11.14.84@KRBTEST.COM
I20260807 18:34:53.357157 23373 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127579900523-11321-0/minicluster-data/master-0/data/instance:
uuid: "aee6188789414d148d67fd2c4e1fb071"
format_stamp: "Formatted at 2026-08-07 18:34:53 on dist-test-slave-6k22"
server_key: "ca0677ca60bd5ddad760b8df2252f71e"
server_key_iv: "eb8e1858f0e9f5a273bce489f3cd1793"
server_key_version: "kuduclusterkey@0"
I20260807 18:34:53.358035 23373 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127579900523-11321-0/minicluster-data/master-0/wal/instance:
uuid: "aee6188789414d148d67fd2c4e1fb071"
format_stamp: "Formatted at 2026-08-07 18:34:53 on dist-test-slave-6k22"
server_key: "ca0677ca60bd5ddad760b8df2252f71e"
server_key_iv: "eb8e1858f0e9f5a273bce489f3cd1793"
server_key_version: "kuduclusterkey@0"
I20260807 18:34:53.363658 23373 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260807 18:34:53.379433 23390 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:34:53.381124 23373 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 18:34:53.381283 23373 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127579900523-11321-0/minicluster-data/master-0/data,/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127579900523-11321-0/minicluster-data/master-0/wal
uuid: "aee6188789414d148d67fd2c4e1fb071"
format_stamp: "Formatted at 2026-08-07 18:34:53 on dist-test-slave-6k22"
server_key: "ca0677ca60bd5ddad760b8df2252f71e"
server_key_iv: "eb8e1858f0e9f5a273bce489f3cd1793"
server_key_version: "kuduclusterkey@0"
I20260807 18:34:53.381443 23373 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127579900523-11321-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127579900523-11321-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127579900523-11321-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:34:53.425123 23373 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:34:53.428582 23373 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:34:53.428920 23373 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:34:53.438733 23373 rpc_server.cc:307] RPC server started. Bound to: 127.11.14.126:45731
I20260807 18:34:53.438759 23444 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.14.126:45731 every 8 connection(s)
I20260807 18:34:53.439985 23373 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127579900523-11321-0/minicluster-data/master-0/data/info.pb
I20260807 18:34:53.443737 23445 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:34:53.449812 11321 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkz56XN/build/debug/bin/kudu as pid 23373
I20260807 18:34:53.449972 11321 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127579900523-11321-0/minicluster-data/master-0/wal/instance
I20260807 18:34:53.450284 11321 external_mini_cluster.cc:1428] Setting key e02c5de04a9777f0fd4a92f50878dd34
I20260807 18:34:53.451723 23445 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aee6188789414d148d67fd2c4e1fb071: Bootstrap starting.
I20260807 18:34:53.454615 23445 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P aee6188789414d148d67fd2c4e1fb071: Neither blocks nor log segments found. Creating new log.
I20260807 18:34:53.455539 23445 log.cc:824] T 00000000000000000000000000000000 P aee6188789414d148d67fd2c4e1fb071: Log is configured to *not* fsync() on all Append() calls
I20260807 18:34:53.457710 23445 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aee6188789414d148d67fd2c4e1fb071: No bootstrap required, opened a new log
Aug 07 18:34:53 dist-test-slave-6k22 krb5kdc[22896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127693, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.14.126@KRBTEST.COM
I20260807 18:34:53.462457 23445 raft_consensus.cc:348] T 00000000000000000000000000000000 P aee6188789414d148d67fd2c4e1fb071 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aee6188789414d148d67fd2c4e1fb071" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 45731 } }
I20260807 18:34:53.462723 23445 raft_consensus.cc:374] T 00000000000000000000000000000000 P aee6188789414d148d67fd2c4e1fb071 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:34:53.462831 23445 raft_consensus.cc:729] T 00000000000000000000000000000000 P aee6188789414d148d67fd2c4e1fb071 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aee6188789414d148d67fd2c4e1fb071, State: Initialized, Role: FOLLOWER
I20260807 18:34:53.463373 23445 consensus_queue.cc:260] T 00000000000000000000000000000000 P aee6188789414d148d67fd2c4e1fb071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aee6188789414d148d67fd2c4e1fb071" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 45731 } }
I20260807 18:34:53.463598 23445 raft_consensus.cc:388] T 00000000000000000000000000000000 P aee6188789414d148d67fd2c4e1fb071 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:34:53.463708 23445 raft_consensus.cc:482] T 00000000000000000000000000000000 P aee6188789414d148d67fd2c4e1fb071 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:34:53.463840 23445 raft_consensus.cc:3037] T 00000000000000000000000000000000 P aee6188789414d148d67fd2c4e1fb071 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:34:53.464957 23445 raft_consensus.cc:504] T 00000000000000000000000000000000 P aee6188789414d148d67fd2c4e1fb071 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aee6188789414d148d67fd2c4e1fb071" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 45731 } }
I20260807 18:34:53.465467 23445 leader_election.cc:302] T 00000000000000000000000000000000 P aee6188789414d148d67fd2c4e1fb071 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: aee6188789414d148d67fd2c4e1fb071; no voters:
I20260807 18:34:53.465976 23445 leader_election.cc:288] T 00000000000000000000000000000000 P aee6188789414d148d67fd2c4e1fb071 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:34:53.466041 23450 raft_consensus.cc:2781] T 00000000000000000000000000000000 P aee6188789414d148d67fd2c4e1fb071 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:34:53.466339 23450 raft_consensus.cc:686] T 00000000000000000000000000000000 P aee6188789414d148d67fd2c4e1fb071 [term 1 LEADER]: Becoming Leader. State: Replica: aee6188789414d148d67fd2c4e1fb071, State: Running, Role: LEADER
I20260807 18:34:53.466707 23450 consensus_queue.cc:237] T 00000000000000000000000000000000 P aee6188789414d148d67fd2c4e1fb071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aee6188789414d148d67fd2c4e1fb071" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 45731 } }
I20260807 18:34:53.466837 23448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:34:53.451925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35180 (local address 127.11.14.126:45731)
0807 18:34:53.452369 (+ 444us) server_negotiation.cc:206] Beginning negotiation
0807 18:34:53.452386 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0807 18:34:53.452424 (+ 38us) server_negotiation.cc:407] Connection header received
0807 18:34:53.453089 (+ 665us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:34:53.453105 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:34:53.453447 (+ 342us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:34:53.453851 (+ 404us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:34:53.454680 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:34:53.455913 (+ 1233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:34:53.457145 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:34:53.457828 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:34:53.457983 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:34:53.461186 (+ 3203us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:34:53.461194 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:34:53.461212 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:34:53.461244 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:34:53.463318 (+ 2074us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:34:53.463884 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:34:53.463890 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:34:53.463897 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:34:53.464031 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:34:53.464377 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:34:53.464380 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:34:53.464382 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:34:53.464924 (+ 542us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:34:53.465150 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0807 18:34:53.465545 (+ 395us) server_negotiation.cc:299] Negotiation successful
0807 18:34:53.465841 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":289,"thread_start_us":134,"threads_started":1}
I20260807 18:34:53.467576 23445 sys_catalog.cc:564] T 00000000000000000000000000000000 P aee6188789414d148d67fd2c4e1fb071 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:34:53.471035 23450 sys_catalog.cc:455] T 00000000000000000000000000000000 P aee6188789414d148d67fd2c4e1fb071 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "aee6188789414d148d67fd2c4e1fb071" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aee6188789414d148d67fd2c4e1fb071" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 45731 } } }
I20260807 18:34:53.471254 23450 sys_catalog.cc:458] T 00000000000000000000000000000000 P aee6188789414d148d67fd2c4e1fb071 [sys.catalog]: This master's current role is: LEADER
I20260807 18:34:53.472293 23445 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127579900523-11321-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260807 18:34:53.473856 23452 sys_catalog.cc:455] T 00000000000000000000000000000000 P aee6188789414d148d67fd2c4e1fb071 [sys.catalog]: SysCatalogTable state changed. Reason: New leader aee6188789414d148d67fd2c4e1fb071. Latest consensus state: current_term: 1 leader_uuid: "aee6188789414d148d67fd2c4e1fb071" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aee6188789414d148d67fd2c4e1fb071" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 45731 } } }
I20260807 18:34:53.473974 23452 sys_catalog.cc:458] T 00000000000000000000000000000000 P aee6188789414d148d67fd2c4e1fb071 [sys.catalog]: This master's current role is: LEADER
I20260807 18:34:53.479321 23464 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:34:53.484929 23464 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:34:53.493075 23464 catalog_manager.cc:1347] Generated new cluster ID: 4428d44dc70e462cbfb3a84a070d3c97
I20260807 18:34:53.493153 23464 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:34:53.506887 23464 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:34:53.508376 23464 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:34:53.516554 23464 catalog_manager.cc:5868] T 00000000000000000000000000000000 P aee6188789414d148d67fd2c4e1fb071: Generated new TSK 0
I20260807 18:34:53.517529 23464 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.14.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.14.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.14.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.65.keytab.
Entry for principal kudu/127.11.14.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.65.keytab.
I20260807 18:34:54.346511 11321 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkz56XN/build/debug/bin/kudu
/tmp/dist-test-taskkz56XN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127579900523-11321-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127579900523-11321-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127579900523-11321-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127579900523-11321-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.14.84:59359/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.14.65:0
--local_ip_for_outbound_sockets=127.11.14.65
--webserver_interface=127.11.14.65
--webserver_port=0
--tserver_master_addrs=127.11.14.126:45731
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.65.keytab
--principal=kudu/127.11.14.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.14.84:38715
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127579900523-11321-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127579900523-11321-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127579900523-11321-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:34:54.487468 23490 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:34:54.487866 23490 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:34:54.487959 23490 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:34:54.493016 23490 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:34:54.493080 23490 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:34:54.493144 23490 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.14.65
I20260807 18:34:54.498958 23490 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.14.84:38715
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127579900523-11321-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127579900523-11321-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.11.14.84:59359/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.65.keytab
--principal=kudu/127.11.14.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.14.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127579900523-11321-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.14.65
--webserver_port=0
--tserver_master_addrs=127.11.14.126:45731
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23490
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.14.65
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127579900523-11321-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:34:54.500080 23490 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:34:54.501139 23490 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:34:54.511897 23495 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:34:54.511906 23498 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:34:54.511910 23496 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:34:54.512861 23490 server_base.cc:1034] running on GCE node
I20260807 18:34:54.513281 23490 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:34:54.514050 23490 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:34:54.515244 23490 hybrid_clock.cc:648] HybridClock initialized: now 1786127694515228 us; error 30 us; skew 500 ppm
Aug 07 18:34:54 dist-test-slave-6k22 krb5kdc[22896](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127694, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:34:54.518430 23490 init.cc:377] Logged in from keytab as kudu/127.11.14.65@KRBTEST.COM (short username kudu)
I20260807 18:34:54.519629 23490 webserver.cc:460] Webserver started at http://127.11.14.65:43115/ using document root <none> and password file <none>
I20260807 18:34:54.520435 23490 fs_manager.cc:359] Metadata directory not provided
I20260807 18:34:54.520543 23490 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:34:54.520799 23490 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 18:34:54 dist-test-slave-6k22 krb5kdc[22896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127694, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.65@KRBTEST.COM for HTTP/127.11.14.84@KRBTEST.COM
I20260807 18:34:54.537443 23490 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127579900523-11321-0/minicluster-data/ts-0/data/instance:
uuid: "5a7c20f774ca4940b145dfb3bab0eeba"
format_stamp: "Formatted at 2026-08-07 18:34:54 on dist-test-slave-6k22"
server_key: "5e5cc04f3d682157a91a7f0593ebd23b"
server_key_iv: "3163dc2f496cfbc5c85fe02a1499480e"
server_key_version: "kuduclusterkey@0"
I20260807 18:34:54.538229 23490 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127579900523-11321-0/minicluster-data/ts-0/wal/instance:
uuid: "5a7c20f774ca4940b145dfb3bab0eeba"
format_stamp: "Formatted at 2026-08-07 18:34:54 on dist-test-slave-6k22"
server_key: "5e5cc04f3d682157a91a7f0593ebd23b"
server_key_iv: "3163dc2f496cfbc5c85fe02a1499480e"
server_key_version: "kuduclusterkey@0"
I20260807 18:34:54.543888 23490 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260807 18:34:54.566231 23507 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:34:54.568297 23490 fs_manager.cc:725] Time spent opening block manager: real 0.007s user 0.004s sys 0.000s
I20260807 18:34:54.568498 23490 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127579900523-11321-0/minicluster-data/ts-0/data,/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127579900523-11321-0/minicluster-data/ts-0/wal
uuid: "5a7c20f774ca4940b145dfb3bab0eeba"
format_stamp: "Formatted at 2026-08-07 18:34:54 on dist-test-slave-6k22"
server_key: "5e5cc04f3d682157a91a7f0593ebd23b"
server_key_iv: "3163dc2f496cfbc5c85fe02a1499480e"
server_key_version: "kuduclusterkey@0"
I20260807 18:34:54.568779 23490 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127579900523-11321-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127579900523-11321-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127579900523-11321-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:34:54.608852 23490 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:34:54.612748 23490 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:34:54.613023 23490 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:34:54.613842 23490 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:34:54.614991 23490 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:34:54.615104 23490 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:34:54.615231 23490 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:34:54.615314 23490 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:34:54.629892 23490 rpc_server.cc:307] RPC server started. Bound to: 127.11.14.65:45283
I20260807 18:34:54.631222 23490 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127579900523-11321-0/minicluster-data/ts-0/data/info.pb
I20260807 18:34:54.632844 23629 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.14.65:45283 every 8 connection(s)
I20260807 18:34:54.634876 11321 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkz56XN/build/debug/bin/kudu as pid 23490
I20260807 18:34:54.635021 11321 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127579900523-11321-0/minicluster-data/ts-0/wal/instance
I20260807 18:34:54.635413 11321 external_mini_cluster.cc:1428] Setting key 7476ea6517420b7d8330552fb9c1f811
Aug 07 18:34:54 dist-test-slave-6k22 krb5kdc[22896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127694, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.65@KRBTEST.COM for kudu/127.11.14.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.11.14.66@KRBTEST.COM; defaulting to no policy
I20260807 18:34:54.652004 23634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:34:54.634129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.65:40827 (local address 127.11.14.126:45731)
0807 18:34:54.635913 (+ 1784us) server_negotiation.cc:206] Beginning negotiation
0807 18:34:54.635917 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:34:54.636385 (+ 468us) server_negotiation.cc:407] Connection header received
0807 18:34:54.637662 (+ 1277us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:34:54.637668 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:34:54.637736 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:34:54.637825 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:34:54.639599 (+ 1774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:34:54.640554 (+ 955us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:34:54.642100 (+ 1546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:34:54.642503 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:34:54.642626 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:34:54.646439 (+ 3813us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:34:54.646445 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:34:54.646450 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:34:54.646485 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:34:54.648703 (+ 2218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:34:54.649497 (+ 794us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:34:54.649501 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:34:54.649503 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:34:54.649602 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:34:54.650082 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:34:54.650088 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:34:54.650092 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:34:54.650371 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:34:54.650474 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0807 18:34:54.651680 (+ 1206us) server_negotiation.cc:299] Negotiation successful
0807 18:34:54.651816 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1696,"thread_start_us":102,"threads_started":1}
I20260807 18:34:54.653576 23635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:34:54.634627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:45731 (local address 127.11.14.65:40827)
0807 18:34:54.636225 (+ 1598us) negotiation.cc:107] Waiting for socket to connect
0807 18:34:54.636285 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0807 18:34:54.637303 (+ 1018us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:34:54.638158 (+ 855us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:34:54.638171 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:34:54.638576 (+ 405us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:34:54.639337 (+ 761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:34:54.639364 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:34:54.640733 (+ 1369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:34:54.640738 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:34:54.641962 (+ 1224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:34:54.641973 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:34:54.642823 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:34:54.642830 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:34:54.643137 (+ 307us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:34:54.644023 (+ 886us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:34:54.644055 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:34:54.646209 (+ 2154us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:34:54.648870 (+ 2661us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:34:54.648879 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:34:54.648899 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:34:54.649323 (+ 424us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:34:54.649743 (+ 420us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:34:54.649746 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:34:54.649748 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:34:54.649956 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:34:54.650537 (+ 581us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:34:54.650544 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:34:54.650879 (+ 335us) client_negotiation.cc:769] Sending connection context
0807 18:34:54.651101 (+ 222us) client_negotiation.cc:240] Negotiation successful
0807 18:34:54.651470 (+ 369us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1317,"thread_start_us":139,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.14.66@KRBTEST.COM" created.
I20260807 18:34:54.655453 23631 heartbeater.cc:347] Connected to a master server at 127.11.14.126:45731
I20260807 18:34:54.655782 23631 heartbeater.cc:464] Registering TS with master...
I20260807 18:34:54.656488 23631 heartbeater.cc:511] Master 127.11.14.126:45731 requested a full tablet report, sending...
I20260807 18:34:54.658411 23407 ts_manager.cc:194] Registered new tserver with Master: 5a7c20f774ca4940b145dfb3bab0eeba (127.11.14.65:45283)
I20260807 18:34:54.660100 23407 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.14.65@KRBTEST.COM'} at 127.11.14.65:40827
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.14.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.66.keytab.
Entry for principal kudu/127.11.14.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.66.keytab.
I20260807 18:34:54.683460 11321 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkz56XN/build/debug/bin/kudu
/tmp/dist-test-taskkz56XN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127579900523-11321-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127579900523-11321-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127579900523-11321-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127579900523-11321-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.14.84:59359/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.14.66:0
--local_ip_for_outbound_sockets=127.11.14.66
--webserver_interface=127.11.14.66
--webserver_port=0
--tserver_master_addrs=127.11.14.126:45731
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.66.keytab
--principal=kudu/127.11.14.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.14.84:38715
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127579900523-11321-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127579900523-11321-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127579900523-11321-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 18:34:54 dist-test-slave-6k22 krb5kdc[22896](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786127694, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260807 18:34:54.865684 23639 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:34:54.866153 23639 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:34:54.866253 23639 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:34:54.872592 23639 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:34:54.872671 23639 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:34:54.872773 23639 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.14.66
I20260807 18:34:54.881174 23639 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.14.84:38715
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127579900523-11321-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127579900523-11321-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.11.14.84:59359/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.66.keytab
--principal=kudu/127.11.14.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.14.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127579900523-11321-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.14.66
--webserver_port=0
--tserver_master_addrs=127.11.14.126:45731
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23639
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.14.66
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127579900523-11321-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:34:54.882609 23639 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:34:54.884115 23639 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:34:54.897490 23646 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:34:54.897524 23645 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:34:54.897812 23648 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:34:54.898696 23639 server_base.cc:1034] running on GCE node
I20260807 18:34:54.899184 23639 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:34:54.900034 23639 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:34:54.901322 23639 hybrid_clock.cc:648] HybridClock initialized: now 1786127694901289 us; error 44 us; skew 500 ppm
Aug 07 18:34:54 dist-test-slave-6k22 krb5kdc[22896](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127694, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:34:54.905314 23639 init.cc:377] Logged in from keytab as kudu/127.11.14.66@KRBTEST.COM (short username kudu)
I20260807 18:34:54.906517 23639 webserver.cc:460] Webserver started at http://127.11.14.66:42717/ using document root <none> and password file <none>
I20260807 18:34:54.907370 23639 fs_manager.cc:359] Metadata directory not provided
I20260807 18:34:54.907473 23639 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:34:54.907791 23639 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 18:34:54 dist-test-slave-6k22 krb5kdc[22896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127694, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.66@KRBTEST.COM for HTTP/127.11.14.84@KRBTEST.COM
I20260807 18:34:54.925494 23639 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127579900523-11321-0/minicluster-data/ts-1/data/instance:
uuid: "317c30312c3444939591d4f9e1e9f02e"
format_stamp: "Formatted at 2026-08-07 18:34:54 on dist-test-slave-6k22"
server_key: "8b335f131621c6dbcd07c498532ca872"
server_key_iv: "f692b217016c6889e205b41461403079"
server_key_version: "kuduclusterkey@0"
I20260807 18:34:54.926628 23639 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127579900523-11321-0/minicluster-data/ts-1/wal/instance:
uuid: "317c30312c3444939591d4f9e1e9f02e"
format_stamp: "Formatted at 2026-08-07 18:34:54 on dist-test-slave-6k22"
server_key: "8b335f131621c6dbcd07c498532ca872"
server_key_iv: "f692b217016c6889e205b41461403079"
server_key_version: "kuduclusterkey@0"
I20260807 18:34:54.932206 23639 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260807 18:34:54.955446 23655 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:34:54.957275 23639 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:34:54.957419 23639 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127579900523-11321-0/minicluster-data/ts-1/data,/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127579900523-11321-0/minicluster-data/ts-1/wal
uuid: "317c30312c3444939591d4f9e1e9f02e"
format_stamp: "Formatted at 2026-08-07 18:34:54 on dist-test-slave-6k22"
server_key: "8b335f131621c6dbcd07c498532ca872"
server_key_iv: "f692b217016c6889e205b41461403079"
server_key_version: "kuduclusterkey@0"
I20260807 18:34:54.957567 23639 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127579900523-11321-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127579900523-11321-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127579900523-11321-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:34:54.980914 23639 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:34:54.985165 23639 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:34:54.985458 23639 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:34:54.986398 23639 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:34:54.987743 23639 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:34:54.987890 23639 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:34:54.988013 23639 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:34:54.988093 23639 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:34:55.013068 23639 rpc_server.cc:307] RPC server started. Bound to: 127.11.14.66:35901
I20260807 18:34:55.014544 23639 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127579900523-11321-0/minicluster-data/ts-1/data/info.pb
I20260807 18:34:55.015800 11321 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkz56XN/build/debug/bin/kudu as pid 23639
I20260807 18:34:55.015944 11321 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127579900523-11321-0/minicluster-data/ts-1/wal/instance
I20260807 18:34:55.016261 11321 external_mini_cluster.cc:1428] Setting key a11975393c0becf1e72deeb279068258
I20260807 18:34:55.018867 23778 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.14.66:35901 every 8 connection(s)
Aug 07 18:34:55 dist-test-slave-6k22 krb5kdc[22896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127694, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.66@KRBTEST.COM for kudu/127.11.14.126@KRBTEST.COM
I20260807 18:34:55.039773 23634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:34:55.022189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.66:39355 (local address 127.11.14.126:45731)
0807 18:34:55.022369 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0807 18:34:55.022375 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:34:55.023182 (+ 807us) server_negotiation.cc:407] Connection header received
0807 18:34:55.024863 (+ 1681us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:34:55.024867 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:34:55.024934 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:34:55.025008 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:34:55.027019 (+ 2011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:34:55.027981 (+ 962us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:34:55.029900 (+ 1919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:34:55.030339 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:34:55.030442 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:34:55.034217 (+ 3775us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:34:55.034222 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:34:55.034226 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:34:55.034259 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:34:55.036518 (+ 2259us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:34:55.037351 (+ 833us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:34:55.037355 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:34:55.037357 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:34:55.037489 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:34:55.038026 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:34:55.038030 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:34:55.038032 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:34:55.038335 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:34:55.038461 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0807 18:34:55.039329 (+ 868us) server_negotiation.cc:299] Negotiation successful
0807 18:34:55.039510 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260807 18:34:55.042037 23782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:34:55.020890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:45731 (local address 127.11.14.66:39355)
0807 18:34:55.022987 (+ 2097us) negotiation.cc:107] Waiting for socket to connect
0807 18:34:55.023052 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0807 18:34:55.024558 (+ 1506us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:34:55.025516 (+ 958us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:34:55.025528 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:34:55.026003 (+ 475us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:34:55.026800 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:34:55.026829 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:34:55.028200 (+ 1371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:34:55.028204 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:34:55.029721 (+ 1517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:34:55.029735 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:34:55.030708 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:34:55.030713 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:34:55.030945 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:34:55.031766 (+ 821us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:34:55.031794 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:34:55.033957 (+ 2163us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:34:55.036708 (+ 2751us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:34:55.036716 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:34:55.036735 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:34:55.037196 (+ 461us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:34:55.037648 (+ 452us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:34:55.037652 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:34:55.037655 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:34:55.037872 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:34:55.038571 (+ 699us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:34:55.038580 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:34:55.038952 (+ 372us) client_negotiation.cc:769] Sending connection context
0807 18:34:55.040401 (+ 1449us) client_negotiation.cc:240] Negotiation successful
0807 18:34:55.040813 (+ 412us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1840,"thread_start_us":218,"threads_started":1}
WARNING: no policy specified for kudu/127.11.14.67@KRBTEST.COM; defaulting to no policy
I20260807 18:34:55.043962 23779 heartbeater.cc:347] Connected to a master server at 127.11.14.126:45731
I20260807 18:34:55.044272 23779 heartbeater.cc:464] Registering TS with master...
I20260807 18:34:55.044973 23779 heartbeater.cc:511] Master 127.11.14.126:45731 requested a full tablet report, sending...
I20260807 18:34:55.046300 23407 ts_manager.cc:194] Registered new tserver with Master: 317c30312c3444939591d4f9e1e9f02e (127.11.14.66:35901)
I20260807 18:34:55.047165 23407 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.14.66@KRBTEST.COM'} at 127.11.14.66:39355
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.14.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.14.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.67.keytab.
Entry for principal kudu/127.11.14.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.67.keytab.
I20260807 18:34:55.092139 11321 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkz56XN/build/debug/bin/kudu
/tmp/dist-test-taskkz56XN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127579900523-11321-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127579900523-11321-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127579900523-11321-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127579900523-11321-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.14.84:59359/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.14.67:0
--local_ip_for_outbound_sockets=127.11.14.67
--webserver_interface=127.11.14.67
--webserver_port=0
--tserver_master_addrs=127.11.14.126:45731
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.67.keytab
--principal=kudu/127.11.14.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.14.84:38715
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127579900523-11321-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127579900523-11321-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127579900523-11321-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:34:55.288296 23784 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:34:55.288726 23784 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:34:55.288818 23784 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:34:55.295583 23784 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:34:55.295686 23784 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:34:55.295800 23784 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.14.67
I20260807 18:34:55.304795 23784 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.14.84:38715
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127579900523-11321-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127579900523-11321-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.11.14.84:59359/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.67.keytab
--principal=kudu/127.11.14.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.14.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127579900523-11321-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.14.67
--webserver_port=0
--tserver_master_addrs=127.11.14.126:45731
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23784
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.14.67
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127579900523-11321-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:34:55.306596 23784 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:34:55.308030 23784 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:34:55.321723 23789 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:34:55.321779 23792 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:34:55.321750 23790 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:34:55.322808 23784 server_base.cc:1034] running on GCE node
I20260807 18:34:55.323375 23784 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:34:55.324158 23784 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:34:55.325363 23784 hybrid_clock.cc:648] HybridClock initialized: now 1786127695325294 us; error 78 us; skew 500 ppm
Aug 07 18:34:55 dist-test-slave-6k22 krb5kdc[22896](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127695, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:34:55.328927 23784 init.cc:377] Logged in from keytab as kudu/127.11.14.67@KRBTEST.COM (short username kudu)
I20260807 18:34:55.329995 23784 webserver.cc:460] Webserver started at http://127.11.14.67:37373/ using document root <none> and password file <none>
I20260807 18:34:55.330677 23784 fs_manager.cc:359] Metadata directory not provided
I20260807 18:34:55.330755 23784 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:34:55.331015 23784 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 18:34:55 dist-test-slave-6k22 krb5kdc[22896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127695, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.67@KRBTEST.COM for HTTP/127.11.14.84@KRBTEST.COM
Aug 07 18:34:55 dist-test-slave-6k22 krb5kdc[22896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127694, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.126@KRBTEST.COM for HTTP/127.11.14.84@KRBTEST.COM
I20260807 18:34:55.349404 23784 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127579900523-11321-0/minicluster-data/ts-2/data/instance:
uuid: "a595b5e3641944aabba56c9680184959"
format_stamp: "Formatted at 2026-08-07 18:34:55 on dist-test-slave-6k22"
server_key: "5e4baaa332a1f4a1380984c63d484f5a"
server_key_iv: "9cde7ae39547e0071b0b10dae0e6392c"
server_key_version: "kuduclusterkey@0"
I20260807 18:34:55.350127 23784 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127579900523-11321-0/minicluster-data/ts-2/wal/instance:
uuid: "a595b5e3641944aabba56c9680184959"
format_stamp: "Formatted at 2026-08-07 18:34:55 on dist-test-slave-6k22"
server_key: "5e4baaa332a1f4a1380984c63d484f5a"
server_key_iv: "9cde7ae39547e0071b0b10dae0e6392c"
server_key_version: "kuduclusterkey@0"
I20260807 18:34:55.365953 23784 fs_manager.cc:691] Time spent creating directory manager: real 0.015s user 0.003s sys 0.003s
I20260807 18:34:55.397564 23800 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:34:55.402935 23784 fs_manager.cc:725] Time spent opening block manager: real 0.022s user 0.002s sys 0.001s
I20260807 18:34:55.403136 23784 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127579900523-11321-0/minicluster-data/ts-2/data,/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127579900523-11321-0/minicluster-data/ts-2/wal
uuid: "a595b5e3641944aabba56c9680184959"
format_stamp: "Formatted at 2026-08-07 18:34:55 on dist-test-slave-6k22"
server_key: "5e4baaa332a1f4a1380984c63d484f5a"
server_key_iv: "9cde7ae39547e0071b0b10dae0e6392c"
server_key_version: "kuduclusterkey@0"
I20260807 18:34:55.403326 23784 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127579900523-11321-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127579900523-11321-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127579900523-11321-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:34:55.431202 23784 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:34:55.441931 23784 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:34:55.442160 23784 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:34:55.442981 23784 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:34:55.444269 23784 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:34:55.444342 23784 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:34:55.444407 23784 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:34:55.444442 23784 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:34:55.470872 23784 rpc_server.cc:307] RPC server started. Bound to: 127.11.14.67:40005
I20260807 18:34:55.472383 23784 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127579900523-11321-0/minicluster-data/ts-2/data/info.pb
I20260807 18:34:55.473513 23926 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.14.67:40005 every 8 connection(s)
I20260807 18:34:55.475771 11321 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkz56XN/build/debug/bin/kudu as pid 23784
I20260807 18:34:55.475905 11321 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127579900523-11321-0/minicluster-data/ts-2/wal/instance
I20260807 18:34:55.476213 11321 external_mini_cluster.cc:1428] Setting key 74618089188bde8b1223aeec17626570
Aug 07 18:34:55 dist-test-slave-6k22 krb5kdc[22896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127695, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.67@KRBTEST.COM for kudu/127.11.14.126@KRBTEST.COM
I20260807 18:34:55.494920 23634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:34:55.478402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.67:51995 (local address 127.11.14.126:45731)
0807 18:34:55.478553 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0807 18:34:55.478558 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:34:55.479387 (+ 829us) server_negotiation.cc:407] Connection header received
0807 18:34:55.480909 (+ 1522us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:34:55.480913 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:34:55.480986 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:34:55.481062 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:34:55.483023 (+ 1961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:34:55.483901 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:34:55.485811 (+ 1910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:34:55.486163 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:34:55.486286 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:34:55.489966 (+ 3680us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:34:55.489971 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:34:55.489974 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:34:55.490019 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:34:55.492015 (+ 1996us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:34:55.492743 (+ 728us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:34:55.492749 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:34:55.492751 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:34:55.492856 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:34:55.493404 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:34:55.493408 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:34:55.493410 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:34:55.493685 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:34:55.493770 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0807 18:34:55.494558 (+ 788us) server_negotiation.cc:299] Negotiation successful
0807 18:34:55.494715 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260807 18:34:55.496783 23929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:34:55.477220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:45731 (local address 127.11.14.67:51995)
0807 18:34:55.479197 (+ 1977us) negotiation.cc:107] Waiting for socket to connect
0807 18:34:55.479262 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0807 18:34:55.480634 (+ 1372us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:34:55.481522 (+ 888us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:34:55.481532 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:34:55.481981 (+ 449us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:34:55.482815 (+ 834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:34:55.482840 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:34:55.484088 (+ 1248us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:34:55.484092 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:34:55.485653 (+ 1561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:34:55.485666 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:34:55.486551 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:34:55.486555 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:34:55.486885 (+ 330us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:34:55.487684 (+ 799us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:34:55.487710 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:34:55.489763 (+ 2053us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:34:55.492175 (+ 2412us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:34:55.492182 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:34:55.492201 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:34:55.492581 (+ 380us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:34:55.493010 (+ 429us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:34:55.493013 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:34:55.493015 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:34:55.493226 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:34:55.493872 (+ 646us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:34:55.493880 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:34:55.494236 (+ 356us) client_negotiation.cc:769] Sending connection context
0807 18:34:55.495189 (+ 953us) client_negotiation.cc:240] Negotiation successful
0807 18:34:55.495614 (+ 425us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1660,"thread_start_us":159,"threads_started":1}
I20260807 18:34:55.498076 23927 heartbeater.cc:347] Connected to a master server at 127.11.14.126:45731
I20260807 18:34:55.498377 23927 heartbeater.cc:464] Registering TS with master...
I20260807 18:34:55.499049 23927 heartbeater.cc:511] Master 127.11.14.126:45731 requested a full tablet report, sending...
I20260807 18:34:55.500495 23407 ts_manager.cc:194] Registered new tserver with Master: a595b5e3641944aabba56c9680184959 (127.11.14.67:40005)
I20260807 18:34:55.501382 23407 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.14.67@KRBTEST.COM'} at 127.11.14.67:51995
I20260807 18:34:55.504851 11321 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 18:34:55.517797 23634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:34:55.507242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35182 (local address 127.11.14.126:45731)
0807 18:34:55.507362 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0807 18:34:55.507366 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:34:55.507940 (+ 574us) server_negotiation.cc:407] Connection header received
0807 18:34:55.508155 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:34:55.508161 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:34:55.508213 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:34:55.508267 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:34:55.509405 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:34:55.510184 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:34:55.511480 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:34:55.511868 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:34:55.511964 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:34:55.514035 (+ 2071us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:34:55.514039 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:34:55.514043 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:34:55.514068 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:34:55.515854 (+ 1786us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:34:55.516391 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:34:55.516394 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:34:55.516396 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:34:55.516485 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:34:55.516834 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:34:55.516837 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:34:55.516839 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:34:55.517039 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:34:55.517168 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0807 18:34:55.517498 (+ 330us) server_negotiation.cc:299] Negotiation successful
0807 18:34:55.517626 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260807 18:34:55.521523 23407 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35182:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 18:34:55.526273 23407 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 18:34:55.546617 23941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:34:55.537494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.65:45283 (local address 127.0.0.1:42328)
0807 18:34:55.539971 (+ 2477us) negotiation.cc:107] Waiting for socket to connect
0807 18:34:55.540036 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0807 18:34:55.540198 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:34:55.541371 (+ 1173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:34:55.541379 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:34:55.541410 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:34:55.541611 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:34:55.541621 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:34:55.543307 (+ 1686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:34:55.543310 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:34:55.544602 (+ 1292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:34:55.544613 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:34:55.545970 (+ 1357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:34:55.545977 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:34:55.546124 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:34:55.546184 (+ 60us) client_negotiation.cc:769] Sending connection context
0807 18:34:55.546310 (+ 126us) client_negotiation.cc:240] Negotiation successful
0807 18:34:55.546438 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2222,"thread_start_us":117,"threads_started":1}
I20260807 18:34:55.548151 23943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:34:55.538361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42328 (local address 127.11.14.65:45283)
0807 18:34:55.540546 (+ 2185us) server_negotiation.cc:206] Beginning negotiation
0807 18:34:55.540557 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 18:34:55.540588 (+ 31us) server_negotiation.cc:407] Connection header received
0807 18:34:55.540699 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:34:55.540705 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:34:55.540958 (+ 253us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:34:55.541143 (+ 185us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:34:55.541812 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:34:55.543139 (+ 1327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:34:55.544757 (+ 1618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:34:55.545619 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:34:55.545730 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:34:55.545904 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0807 18:34:55.547867 (+ 1963us) server_negotiation.cc:299] Negotiation successful
0807 18:34:55.547982 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2054,"thread_start_us":88,"threads_started":1}
I20260807 18:34:55.551980 23557 tablet_service.cc:1467] Processing CreateTablet for tablet 263d652ffd924f3f939165f8fdb413c4 (DEFAULT_TABLE table=test-table [id=e930a37adec1430ca4253f5cbd68e183]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:34:55.553092 23557 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 263d652ffd924f3f939165f8fdb413c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:34:55.557260 23942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:34:55.538466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.67:40005 (local address 127.0.0.1:50198)
0807 18:34:55.544874 (+ 6408us) negotiation.cc:107] Waiting for socket to connect
0807 18:34:55.544889 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 18:34:55.544985 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:34:55.545786 (+ 801us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:34:55.545789 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:34:55.545808 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:34:55.545899 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:34:55.545906 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:34:55.547290 (+ 1384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:34:55.547295 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:34:55.548483 (+ 1188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:34:55.548494 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:34:55.556726 (+ 8232us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:34:55.556731 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:34:55.556874 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:34:55.556906 (+ 32us) client_negotiation.cc:769] Sending connection context
0807 18:34:55.556999 (+ 93us) client_negotiation.cc:240] Negotiation successful
0807 18:34:55.557084 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6341,"thread_start_us":151,"threads_started":1}
I20260807 18:34:55.557868 23946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:34:55.539538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50198 (local address 127.11.14.67:40005)
0807 18:34:55.540482 (+ 944us) server_negotiation.cc:206] Beginning negotiation
0807 18:34:55.540493 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 18:34:55.545106 (+ 4613us) server_negotiation.cc:407] Connection header received
0807 18:34:55.545200 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:34:55.545206 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:34:55.545424 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:34:55.545605 (+ 181us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:34:55.546044 (+ 439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:34:55.547157 (+ 1113us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:34:55.548664 (+ 1507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:34:55.549312 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:34:55.549429 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:34:55.549576 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0807 18:34:55.557582 (+ 8006us) server_negotiation.cc:299] Negotiation successful
0807 18:34:55.557707 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":845,"thread_start_us":93,"threads_started":1}
I20260807 18:34:55.561192 23855 tablet_service.cc:1467] Processing CreateTablet for tablet 263d652ffd924f3f939165f8fdb413c4 (DEFAULT_TABLE table=test-table [id=e930a37adec1430ca4253f5cbd68e183]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:34:55.562284 23855 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 263d652ffd924f3f939165f8fdb413c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:34:55.569357 23944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:34:55.537503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.66:35901 (local address 127.0.0.1:45276)
0807 18:34:55.540125 (+ 2622us) negotiation.cc:107] Waiting for socket to connect
0807 18:34:55.540139 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 18:34:55.540231 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:34:55.541984 (+ 1753us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:34:55.541987 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:34:55.542004 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:34:55.542092 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:34:55.542099 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:34:55.543449 (+ 1350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:34:55.543453 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:34:55.563206 (+ 19753us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:34:55.563218 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:34:55.568897 (+ 5679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:34:55.568901 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:34:55.569004 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:34:55.569045 (+ 41us) client_negotiation.cc:769] Sending connection context
0807 18:34:55.569104 (+ 59us) client_negotiation.cc:240] Negotiation successful
0807 18:34:55.569211 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2562,"thread_start_us":203,"threads_started":1}
I20260807 18:34:55.570493 23945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:34:55.539213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45276 (local address 127.11.14.66:35901)
0807 18:34:55.539986 (+ 773us) server_negotiation.cc:206] Beginning negotiation
0807 18:34:55.540003 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0807 18:34:55.540536 (+ 533us) server_negotiation.cc:407] Connection header received
0807 18:34:55.540620 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:34:55.540626 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:34:55.540851 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:34:55.541806 (+ 955us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:34:55.542232 (+ 426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:34:55.543319 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:34:55.567656 (+ 24337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:34:55.568613 (+ 957us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:34:55.569699 (+ 1086us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:34:55.569862 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0807 18:34:55.570265 (+ 403us) server_negotiation.cc:299] Negotiation successful
0807 18:34:55.570366 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":661,"thread_start_us":86,"threads_started":1}
I20260807 18:34:55.573374 23697 tablet_service.cc:1467] Processing CreateTablet for tablet 263d652ffd924f3f939165f8fdb413c4 (DEFAULT_TABLE table=test-table [id=e930a37adec1430ca4253f5cbd68e183]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:34:55.574501 23697 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 263d652ffd924f3f939165f8fdb413c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:34:55.579993 23947 tablet_bootstrap.cc:492] T 263d652ffd924f3f939165f8fdb413c4 P 5a7c20f774ca4940b145dfb3bab0eeba: Bootstrap starting.
I20260807 18:34:55.582597 23947 tablet_bootstrap.cc:654] T 263d652ffd924f3f939165f8fdb413c4 P 5a7c20f774ca4940b145dfb3bab0eeba: Neither blocks nor log segments found. Creating new log.
I20260807 18:34:55.583017 23949 tablet_bootstrap.cc:492] T 263d652ffd924f3f939165f8fdb413c4 P 317c30312c3444939591d4f9e1e9f02e: Bootstrap starting.
I20260807 18:34:55.583593 23947 log.cc:824] T 263d652ffd924f3f939165f8fdb413c4 P 5a7c20f774ca4940b145dfb3bab0eeba: Log is configured to *not* fsync() on all Append() calls
I20260807 18:34:55.584503 23948 tablet_bootstrap.cc:492] T 263d652ffd924f3f939165f8fdb413c4 P a595b5e3641944aabba56c9680184959: Bootstrap starting.
I20260807 18:34:55.585340 23949 tablet_bootstrap.cc:654] T 263d652ffd924f3f939165f8fdb413c4 P 317c30312c3444939591d4f9e1e9f02e: Neither blocks nor log segments found. Creating new log.
I20260807 18:34:55.586246 23949 log.cc:824] T 263d652ffd924f3f939165f8fdb413c4 P 317c30312c3444939591d4f9e1e9f02e: Log is configured to *not* fsync() on all Append() calls
I20260807 18:34:55.587445 23948 tablet_bootstrap.cc:654] T 263d652ffd924f3f939165f8fdb413c4 P a595b5e3641944aabba56c9680184959: Neither blocks nor log segments found. Creating new log.
I20260807 18:34:55.588374 23948 log.cc:824] T 263d652ffd924f3f939165f8fdb413c4 P a595b5e3641944aabba56c9680184959: Log is configured to *not* fsync() on all Append() calls
I20260807 18:34:55.588593 23949 tablet_bootstrap.cc:492] T 263d652ffd924f3f939165f8fdb413c4 P 317c30312c3444939591d4f9e1e9f02e: No bootstrap required, opened a new log
I20260807 18:34:55.588852 23949 ts_tablet_manager.cc:1397] T 263d652ffd924f3f939165f8fdb413c4 P 317c30312c3444939591d4f9e1e9f02e: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260807 18:34:55.590689 23947 tablet_bootstrap.cc:492] T 263d652ffd924f3f939165f8fdb413c4 P 5a7c20f774ca4940b145dfb3bab0eeba: No bootstrap required, opened a new log
I20260807 18:34:55.590893 23947 ts_tablet_manager.cc:1397] T 263d652ffd924f3f939165f8fdb413c4 P 5a7c20f774ca4940b145dfb3bab0eeba: Time spent bootstrapping tablet: real 0.011s user 0.004s sys 0.000s
I20260807 18:34:55.593897 23949 raft_consensus.cc:348] T 263d652ffd924f3f939165f8fdb413c4 P 317c30312c3444939591d4f9e1e9f02e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "317c30312c3444939591d4f9e1e9f02e" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 35901 } } peers { permanent_uuid: "a595b5e3641944aabba56c9680184959" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 40005 } } peers { permanent_uuid: "5a7c20f774ca4940b145dfb3bab0eeba" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 45283 } }
I20260807 18:34:55.594214 23949 raft_consensus.cc:374] T 263d652ffd924f3f939165f8fdb413c4 P 317c30312c3444939591d4f9e1e9f02e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:34:55.594273 23949 raft_consensus.cc:729] T 263d652ffd924f3f939165f8fdb413c4 P 317c30312c3444939591d4f9e1e9f02e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 317c30312c3444939591d4f9e1e9f02e, State: Initialized, Role: FOLLOWER
I20260807 18:34:55.594822 23949 consensus_queue.cc:260] T 263d652ffd924f3f939165f8fdb413c4 P 317c30312c3444939591d4f9e1e9f02e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "317c30312c3444939591d4f9e1e9f02e" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 35901 } } peers { permanent_uuid: "a595b5e3641944aabba56c9680184959" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 40005 } } peers { permanent_uuid: "5a7c20f774ca4940b145dfb3bab0eeba" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 45283 } }
I20260807 18:34:55.595947 23949 ts_tablet_manager.cc:1428] T 263d652ffd924f3f939165f8fdb413c4 P 317c30312c3444939591d4f9e1e9f02e: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20260807 18:34:55.596349 23779 heartbeater.cc:503] Master 127.11.14.126:45731 was elected leader, sending a full tablet report...
I20260807 18:34:55.595497 23947 raft_consensus.cc:348] T 263d652ffd924f3f939165f8fdb413c4 P 5a7c20f774ca4940b145dfb3bab0eeba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "317c30312c3444939591d4f9e1e9f02e" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 35901 } } peers { permanent_uuid: "a595b5e3641944aabba56c9680184959" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 40005 } } peers { permanent_uuid: "5a7c20f774ca4940b145dfb3bab0eeba" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 45283 } }
I20260807 18:34:55.603010 23947 raft_consensus.cc:374] T 263d652ffd924f3f939165f8fdb413c4 P 5a7c20f774ca4940b145dfb3bab0eeba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:34:55.603083 23947 raft_consensus.cc:729] T 263d652ffd924f3f939165f8fdb413c4 P 5a7c20f774ca4940b145dfb3bab0eeba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a7c20f774ca4940b145dfb3bab0eeba, State: Initialized, Role: FOLLOWER
I20260807 18:34:55.603648 23947 consensus_queue.cc:260] T 263d652ffd924f3f939165f8fdb413c4 P 5a7c20f774ca4940b145dfb3bab0eeba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "317c30312c3444939591d4f9e1e9f02e" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 35901 } } peers { permanent_uuid: "a595b5e3641944aabba56c9680184959" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 40005 } } peers { permanent_uuid: "5a7c20f774ca4940b145dfb3bab0eeba" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 45283 } }
I20260807 18:34:55.604748 23947 ts_tablet_manager.cc:1428] T 263d652ffd924f3f939165f8fdb413c4 P 5a7c20f774ca4940b145dfb3bab0eeba: Time spent starting tablet: real 0.014s user 0.007s sys 0.000s
I20260807 18:34:55.605113 23631 heartbeater.cc:503] Master 127.11.14.126:45731 was elected leader, sending a full tablet report...
I20260807 18:34:55.607263 23948 tablet_bootstrap.cc:492] T 263d652ffd924f3f939165f8fdb413c4 P a595b5e3641944aabba56c9680184959: No bootstrap required, opened a new log
I20260807 18:34:55.607506 23948 ts_tablet_manager.cc:1397] T 263d652ffd924f3f939165f8fdb413c4 P a595b5e3641944aabba56c9680184959: Time spent bootstrapping tablet: real 0.023s user 0.004s sys 0.000s
I20260807 18:34:55.611779 23948 raft_consensus.cc:348] T 263d652ffd924f3f939165f8fdb413c4 P a595b5e3641944aabba56c9680184959 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "317c30312c3444939591d4f9e1e9f02e" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 35901 } } peers { permanent_uuid: "a595b5e3641944aabba56c9680184959" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 40005 } } peers { permanent_uuid: "5a7c20f774ca4940b145dfb3bab0eeba" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 45283 } }
I20260807 18:34:55.612064 23948 raft_consensus.cc:374] T 263d652ffd924f3f939165f8fdb413c4 P a595b5e3641944aabba56c9680184959 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:34:55.612147 23948 raft_consensus.cc:729] T 263d652ffd924f3f939165f8fdb413c4 P a595b5e3641944aabba56c9680184959 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a595b5e3641944aabba56c9680184959, State: Initialized, Role: FOLLOWER
I20260807 18:34:55.612645 23948 consensus_queue.cc:260] T 263d652ffd924f3f939165f8fdb413c4 P a595b5e3641944aabba56c9680184959 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "317c30312c3444939591d4f9e1e9f02e" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 35901 } } peers { permanent_uuid: "a595b5e3641944aabba56c9680184959" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 40005 } } peers { permanent_uuid: "5a7c20f774ca4940b145dfb3bab0eeba" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 45283 } }
I20260807 18:34:55.613765 23948 ts_tablet_manager.cc:1428] T 263d652ffd924f3f939165f8fdb413c4 P a595b5e3641944aabba56c9680184959: Time spent starting tablet: real 0.006s user 0.003s sys 0.002s
I20260807 18:34:55.614135 23927 heartbeater.cc:503] Master 127.11.14.126:45731 was elected leader, sending a full tablet report...
W20260807 18:34:55.635403 23632 tablet.cc:2369] T 263d652ffd924f3f939165f8fdb413c4 P 5a7c20f774ca4940b145dfb3bab0eeba: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:34:55.690245 23956 raft_consensus.cc:482] T 263d652ffd924f3f939165f8fdb413c4 P a595b5e3641944aabba56c9680184959 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:34:55.690565 23956 raft_consensus.cc:504] T 263d652ffd924f3f939165f8fdb413c4 P a595b5e3641944aabba56c9680184959 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "317c30312c3444939591d4f9e1e9f02e" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 35901 } } peers { permanent_uuid: "a595b5e3641944aabba56c9680184959" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 40005 } } peers { permanent_uuid: "5a7c20f774ca4940b145dfb3bab0eeba" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 45283 } }
I20260807 18:34:55.692049 23956 leader_election.cc:288] T 263d652ffd924f3f939165f8fdb413c4 P a595b5e3641944aabba56c9680184959 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 317c30312c3444939591d4f9e1e9f02e (127.11.14.66:35901), 5a7c20f774ca4940b145dfb3bab0eeba (127.11.14.65:45283)
I20260807 18:34:55.697233 23929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:34:55.692281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.66:35901 (local address 127.11.14.67:43461)
0807 18:34:55.692988 (+ 707us) negotiation.cc:107] Waiting for socket to connect
0807 18:34:55.693005 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 18:34:55.693094 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:34:55.693410 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:34:55.693413 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:34:55.693434 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:34:55.693564 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:34:55.693570 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:34:55.694597 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:34:55.694601 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:34:55.695844 (+ 1243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:34:55.695857 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:34:55.696824 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:34:55.696827 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:34:55.696930 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:34:55.696952 (+ 22us) client_negotiation.cc:769] Sending connection context
0807 18:34:55.697020 (+ 68us) client_negotiation.cc:240] Negotiation successful
0807 18:34:55.697095 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":625}
I20260807 18:34:55.698225 23945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:34:55.693034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.67:43461 (local address 127.11.14.66:35901)
0807 18:34:55.693160 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0807 18:34:55.693164 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:34:55.693179 (+ 15us) server_negotiation.cc:407] Connection header received
0807 18:34:55.693228 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:34:55.693230 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:34:55.693304 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:34:55.693397 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:34:55.693708 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:34:55.694458 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:34:55.695990 (+ 1532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:34:55.696578 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:34:55.696704 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:34:55.696759 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0807 18:34:55.698018 (+ 1259us) server_negotiation.cc:299] Negotiation successful
0807 18:34:55.698103 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260807 18:34:55.698930 23720 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "263d652ffd924f3f939165f8fdb413c4" candidate_uuid: "a595b5e3641944aabba56c9680184959" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "317c30312c3444939591d4f9e1e9f02e" is_pre_election: true
I20260807 18:34:55.699071 23955 raft_consensus.cc:482] T 263d652ffd924f3f939165f8fdb413c4 P 5a7c20f774ca4940b145dfb3bab0eeba [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:34:55.699291 23720 raft_consensus.cc:2454] T 263d652ffd924f3f939165f8fdb413c4 P 317c30312c3444939591d4f9e1e9f02e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a595b5e3641944aabba56c9680184959 in term 0.
I20260807 18:34:55.699295 23955 raft_consensus.cc:504] T 263d652ffd924f3f939165f8fdb413c4 P 5a7c20f774ca4940b145dfb3bab0eeba [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "317c30312c3444939591d4f9e1e9f02e" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 35901 } } peers { permanent_uuid: "a595b5e3641944aabba56c9680184959" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 40005 } } peers { permanent_uuid: "5a7c20f774ca4940b145dfb3bab0eeba" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 45283 } }
I20260807 18:34:55.700029 23803 leader_election.cc:302] T 263d652ffd924f3f939165f8fdb413c4 P a595b5e3641944aabba56c9680184959 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 317c30312c3444939591d4f9e1e9f02e, a595b5e3641944aabba56c9680184959; no voters:
I20260807 18:34:55.700404 23956 raft_consensus.cc:2781] T 263d652ffd924f3f939165f8fdb413c4 P a595b5e3641944aabba56c9680184959 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:34:55.700513 23956 raft_consensus.cc:482] T 263d652ffd924f3f939165f8fdb413c4 P a595b5e3641944aabba56c9680184959 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:34:55.700572 23956 raft_consensus.cc:3037] T 263d652ffd924f3f939165f8fdb413c4 P a595b5e3641944aabba56c9680184959 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:34:55.700673 23955 leader_election.cc:288] T 263d652ffd924f3f939165f8fdb413c4 P 5a7c20f774ca4940b145dfb3bab0eeba [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 317c30312c3444939591d4f9e1e9f02e (127.11.14.66:35901), a595b5e3641944aabba56c9680184959 (127.11.14.67:40005)
I20260807 18:34:55.702164 23956 raft_consensus.cc:504] T 263d652ffd924f3f939165f8fdb413c4 P a595b5e3641944aabba56c9680184959 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "317c30312c3444939591d4f9e1e9f02e" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 35901 } } peers { permanent_uuid: "a595b5e3641944aabba56c9680184959" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 40005 } } peers { permanent_uuid: "5a7c20f774ca4940b145dfb3bab0eeba" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 45283 } }
I20260807 18:34:55.702515 23956 leader_election.cc:288] T 263d652ffd924f3f939165f8fdb413c4 P a595b5e3641944aabba56c9680184959 [CANDIDATE]: Term 1 election: Requested vote from peers 317c30312c3444939591d4f9e1e9f02e (127.11.14.66:35901), 5a7c20f774ca4940b145dfb3bab0eeba (127.11.14.65:45283)
I20260807 18:34:55.703274 23720 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "263d652ffd924f3f939165f8fdb413c4" candidate_uuid: "a595b5e3641944aabba56c9680184959" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "317c30312c3444939591d4f9e1e9f02e"
I20260807 18:34:55.703399 23720 raft_consensus.cc:3037] T 263d652ffd924f3f939165f8fdb413c4 P 317c30312c3444939591d4f9e1e9f02e [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:34:55.704833 23720 raft_consensus.cc:2454] T 263d652ffd924f3f939165f8fdb413c4 P 317c30312c3444939591d4f9e1e9f02e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a595b5e3641944aabba56c9680184959 in term 1.
I20260807 18:34:55.705231 23803 leader_election.cc:302] T 263d652ffd924f3f939165f8fdb413c4 P a595b5e3641944aabba56c9680184959 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 317c30312c3444939591d4f9e1e9f02e, a595b5e3641944aabba56c9680184959; no voters:
I20260807 18:34:55.705405 23956 raft_consensus.cc:2781] T 263d652ffd924f3f939165f8fdb413c4 P a595b5e3641944aabba56c9680184959 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:34:55.705615 23956 raft_consensus.cc:686] T 263d652ffd924f3f939165f8fdb413c4 P a595b5e3641944aabba56c9680184959 [term 1 LEADER]: Becoming Leader. State: Replica: a595b5e3641944aabba56c9680184959, State: Running, Role: LEADER
I20260807 18:34:55.706019 23956 consensus_queue.cc:237] T 263d652ffd924f3f939165f8fdb413c4 P a595b5e3641944aabba56c9680184959 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "317c30312c3444939591d4f9e1e9f02e" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 35901 } } peers { permanent_uuid: "a595b5e3641944aabba56c9680184959" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 40005 } } peers { permanent_uuid: "5a7c20f774ca4940b145dfb3bab0eeba" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 45283 } }
I20260807 18:34:55.710505 23929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:34:55.692548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.65:45283 (local address 127.11.14.67:39485)
0807 18:34:55.697399 (+ 4851us) negotiation.cc:107] Waiting for socket to connect
0807 18:34:55.697413 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 18:34:55.697494 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:34:55.698651 (+ 1157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:34:55.698654 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:34:55.698674 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:34:55.698763 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:34:55.698768 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:34:55.707874 (+ 9106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:34:55.707878 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:34:55.708966 (+ 1088us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:34:55.708979 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:34:55.710089 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:34:55.710093 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:34:55.710183 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:34:55.710207 (+ 24us) client_negotiation.cc:769] Sending connection context
0807 18:34:55.710276 (+ 69us) client_negotiation.cc:240] Negotiation successful
0807 18:34:55.710349 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4798,"thread_start_us":142,"threads_started":1}
I20260807 18:34:55.710960 23943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:34:55.693359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.67:39485 (local address 127.11.14.65:45283)
0807 18:34:55.693483 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0807 18:34:55.693488 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:34:55.697580 (+ 4092us) server_negotiation.cc:407] Connection header received
0807 18:34:55.697644 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:34:55.697646 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:34:55.697708 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:34:55.697779 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:34:55.698900 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:34:55.699610 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:34:55.709161 (+ 9551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:34:55.709824 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:34:55.709942 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:34:55.709994 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0807 18:34:55.710764 (+ 770us) server_negotiation.cc:299] Negotiation successful
0807 18:34:55.710845 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260807 18:34:55.711547 23563 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "263d652ffd924f3f939165f8fdb413c4" candidate_uuid: "a595b5e3641944aabba56c9680184959" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a7c20f774ca4940b145dfb3bab0eeba" is_pre_election: true
I20260807 18:34:55.711833 23563 raft_consensus.cc:2454] T 263d652ffd924f3f939165f8fdb413c4 P 5a7c20f774ca4940b145dfb3bab0eeba [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a595b5e3641944aabba56c9680184959 in term 0.
I20260807 18:34:55.712227 23578 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "263d652ffd924f3f939165f8fdb413c4" candidate_uuid: "a595b5e3641944aabba56c9680184959" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a7c20f774ca4940b145dfb3bab0eeba"
I20260807 18:34:55.712340 23578 raft_consensus.cc:3037] T 263d652ffd924f3f939165f8fdb413c4 P 5a7c20f774ca4940b145dfb3bab0eeba [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:34:55.713497 23407 catalog_manager.cc:5495] T 263d652ffd924f3f939165f8fdb413c4 P a595b5e3641944aabba56c9680184959 reported cstate change: term changed from 0 to 1, leader changed from <none> to a595b5e3641944aabba56c9680184959 (127.11.14.67). New cstate: current_term: 1 leader_uuid: "a595b5e3641944aabba56c9680184959" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "317c30312c3444939591d4f9e1e9f02e" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 35901 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a595b5e3641944aabba56c9680184959" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 40005 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5a7c20f774ca4940b145dfb3bab0eeba" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 45283 } health_report { overall_health: UNKNOWN } } }
I20260807 18:34:55.713904 23578 raft_consensus.cc:2454] T 263d652ffd924f3f939165f8fdb413c4 P 5a7c20f774ca4940b145dfb3bab0eeba [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a595b5e3641944aabba56c9680184959 in term 1.
I20260807 18:34:55.716221 23946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:34:55.702927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.65:40767 (local address 127.11.14.67:40005)
0807 18:34:55.703043 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0807 18:34:55.703049 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:34:55.712186 (+ 9137us) server_negotiation.cc:407] Connection header received
0807 18:34:55.712367 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:34:55.712370 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:34:55.712407 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:34:55.712451 (+ 44us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:34:55.712893 (+ 442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:34:55.713524 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:34:55.714938 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:34:55.715482 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:34:55.715598 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:34:55.715647 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0807 18:34:55.716032 (+ 385us) server_negotiation.cc:299] Negotiation successful
0807 18:34:55.716107 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260807 18:34:55.716560 23960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:34:55.701367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.67:40005 (local address 127.11.14.65:40767)
0807 18:34:55.712106 (+ 10739us) negotiation.cc:107] Waiting for socket to connect
0807 18:34:55.712119 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 18:34:55.712259 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:34:55.712599 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:34:55.712602 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:34:55.712620 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:34:55.712768 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:34:55.712775 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:34:55.713682 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:34:55.713685 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:34:55.714796 (+ 1111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:34:55.714808 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:34:55.715753 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:34:55.715756 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:34:55.715861 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:34:55.715884 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 18:34:55.716340 (+ 456us) client_negotiation.cc:240] Negotiation successful
0807 18:34:55.716411 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10681,"thread_start_us":100,"threads_started":1}
I20260807 18:34:55.717598 23869 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "263d652ffd924f3f939165f8fdb413c4" candidate_uuid: "5a7c20f774ca4940b145dfb3bab0eeba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a595b5e3641944aabba56c9680184959" is_pre_election: true
I20260807 18:34:55.722436 23959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:34:55.701010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.66:35901 (local address 127.11.14.65:59431)
0807 18:34:55.711755 (+ 10745us) negotiation.cc:107] Waiting for socket to connect
0807 18:34:55.711772 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 18:34:55.711879 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:34:55.718727 (+ 6848us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:34:55.718730 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:34:55.718748 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:34:55.718866 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:34:55.718871 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:34:55.719859 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:34:55.719863 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:34:55.721099 (+ 1236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:34:55.721110 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:34:55.722033 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:34:55.722036 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:34:55.722147 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:34:55.722175 (+ 28us) client_negotiation.cc:769] Sending connection context
0807 18:34:55.722230 (+ 55us) client_negotiation.cc:240] Negotiation successful
0807 18:34:55.722298 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10654,"thread_start_us":127,"threads_started":1}
I20260807 18:34:55.722929 23945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:34:55.702782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.65:59431 (local address 127.11.14.66:35901)
0807 18:34:55.707800 (+ 5018us) server_negotiation.cc:206] Beginning negotiation
0807 18:34:55.707804 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:34:55.711879 (+ 4075us) server_negotiation.cc:407] Connection header received
0807 18:34:55.718482 (+ 6603us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:34:55.718485 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:34:55.718538 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:34:55.718614 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:34:55.718985 (+ 371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:34:55.719732 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:34:55.721256 (+ 1524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:34:55.721866 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:34:55.722662 (+ 796us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:34:55.722714 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0807 18:34:55.722752 (+ 38us) server_negotiation.cc:299] Negotiation successful
0807 18:34:55.722816 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4929}
I20260807 18:34:55.723305 23720 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "263d652ffd924f3f939165f8fdb413c4" candidate_uuid: "5a7c20f774ca4940b145dfb3bab0eeba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "317c30312c3444939591d4f9e1e9f02e" is_pre_election: true
I20260807 18:34:55.723515 23720 raft_consensus.cc:2379] T 263d652ffd924f3f939165f8fdb413c4 P 317c30312c3444939591d4f9e1e9f02e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a7c20f774ca4940b145dfb3bab0eeba in current term 1: Already voted for candidate a595b5e3641944aabba56c9680184959 in this term.
I20260807 18:34:55.724088 23509 leader_election.cc:302] T 263d652ffd924f3f939165f8fdb413c4 P 5a7c20f774ca4940b145dfb3bab0eeba [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a7c20f774ca4940b145dfb3bab0eeba; no voters: 317c30312c3444939591d4f9e1e9f02e, a595b5e3641944aabba56c9680184959
I20260807 18:34:55.724601 23955 raft_consensus.cc:2726] T 263d652ffd924f3f939165f8fdb413c4 P 5a7c20f774ca4940b145dfb3bab0eeba [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260807 18:34:55.724795 23928 tablet.cc:2369] T 263d652ffd924f3f939165f8fdb413c4 P a595b5e3641944aabba56c9680184959: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:34:55.743475 23946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:34:55.736508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50214 (local address 127.11.14.67:40005)
0807 18:34:55.737752 (+ 1244us) server_negotiation.cc:206] Beginning negotiation
0807 18:34:55.737758 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:34:55.737773 (+ 15us) server_negotiation.cc:407] Connection header received
0807 18:34:55.737875 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:34:55.737878 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:34:55.737943 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:34:55.738055 (+ 112us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:34:55.738543 (+ 488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:34:55.739336 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:34:55.740638 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:34:55.740994 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:34:55.741096 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:34:55.741723 (+ 627us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:34:55.742337 (+ 614us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:34:55.742464 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0807 18:34:55.743247 (+ 783us) server_negotiation.cc:299] Negotiation successful
0807 18:34:55.743306 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1150}
Aug 07 18:34:55 dist-test-slave-6k22 krb5kdc[22896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127694, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.126@KRBTEST.COM for HTTP/127.11.14.84@KRBTEST.COM
I20260807 18:34:55.756251 23578 raft_consensus.cc:1261] T 263d652ffd924f3f939165f8fdb413c4 P 5a7c20f774ca4940b145dfb3bab0eeba [term 1 FOLLOWER]: Refusing update from remote peer a595b5e3641944aabba56c9680184959: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:34:55.756467 23720 raft_consensus.cc:1261] T 263d652ffd924f3f939165f8fdb413c4 P 317c30312c3444939591d4f9e1e9f02e [term 1 FOLLOWER]: Refusing update from remote peer a595b5e3641944aabba56c9680184959: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:34:55.756970 23956 consensus_queue.cc:1035] T 263d652ffd924f3f939165f8fdb413c4 P a595b5e3641944aabba56c9680184959 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a7c20f774ca4940b145dfb3bab0eeba" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 45283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:34:55.757174 23961 consensus_queue.cc:1035] T 263d652ffd924f3f939165f8fdb413c4 P a595b5e3641944aabba56c9680184959 [LEADER]: Connected to new peer: Peer: permanent_uuid: "317c30312c3444939591d4f9e1e9f02e" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 35901 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260807 18:34:55.769600 23780 tablet.cc:2369] T 263d652ffd924f3f939165f8fdb413c4 P 317c30312c3444939591d4f9e1e9f02e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:34:55.777878 23969 mvcc.cc:205] Tried to move back new op lower bound from 7315979041807532032 to 7315979041614835712. Current Snapshot: MvccSnapshot[applied={T|T < 7315979041807532032}]
I20260807 18:34:55.785612 23968 mvcc.cc:205] Tried to move back new op lower bound from 7315979041807532032 to 7315979041614835712. Current Snapshot: MvccSnapshot[applied={T|T < 7315979041807532032}]
I20260807 18:34:55.790923 23405 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35182:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:34:55.791129 23405 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35182:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:34:55.793721 23405 catalog_manager.cc:5782] T 00000000000000000000000000000000 P aee6188789414d148d67fd2c4e1fb071: Sending DeleteTablet for 3 replicas of tablet 263d652ffd924f3f939165f8fdb413c4
I20260807 18:34:55.794730 23855 tablet_service.cc:1514] Processing DeleteTablet for tablet 263d652ffd924f3f939165f8fdb413c4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:34:55 UTC) from {username='kudu', principal='kudu/127.11.14.126@KRBTEST.COM'} at 127.0.0.1:50198
I20260807 18:34:55.795423 23697 tablet_service.cc:1514] Processing DeleteTablet for tablet 263d652ffd924f3f939165f8fdb413c4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:34:55 UTC) from {username='kudu', principal='kudu/127.11.14.126@KRBTEST.COM'} at 127.0.0.1:45276
I20260807 18:34:55.796037 23557 tablet_service.cc:1514] Processing DeleteTablet for tablet 263d652ffd924f3f939165f8fdb413c4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:34:55 UTC) from {username='kudu', principal='kudu/127.11.14.126@KRBTEST.COM'} at 127.0.0.1:42328
I20260807 18:34:55.797154 11321 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkz56XN/build/debug/bin/kudu with pid 23490
I20260807 18:34:55.807813 23977 tablet_replica.cc:330] T 263d652ffd924f3f939165f8fdb413c4 P 317c30312c3444939591d4f9e1e9f02e: stopping tablet replica
I20260807 18:34:55.811367 23977 raft_consensus.cc:2229] T 263d652ffd924f3f939165f8fdb413c4 P 317c30312c3444939591d4f9e1e9f02e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:34:55.811800 23977 raft_consensus.cc:2258] T 263d652ffd924f3f939165f8fdb413c4 P 317c30312c3444939591d4f9e1e9f02e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:34:55.812704 23976 tablet_replica.cc:330] T 263d652ffd924f3f939165f8fdb413c4 P a595b5e3641944aabba56c9680184959: stopping tablet replica
I20260807 18:34:55.813011 23976 raft_consensus.cc:2229] T 263d652ffd924f3f939165f8fdb413c4 P a595b5e3641944aabba56c9680184959 [term 1 LEADER]: Raft consensus shutting down.
I20260807 18:34:55.813395 23976 raft_consensus.cc:2258] T 263d652ffd924f3f939165f8fdb413c4 P a595b5e3641944aabba56c9680184959 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:34:55.813822 23977 ts_tablet_manager.cc:1905] T 263d652ffd924f3f939165f8fdb413c4 P 317c30312c3444939591d4f9e1e9f02e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:34:55.817651 23976 ts_tablet_manager.cc:1905] T 263d652ffd924f3f939165f8fdb413c4 P a595b5e3641944aabba56c9680184959: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260807 18:34:55.818786 23393 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.14.65:45283 (error 108)
I20260807 18:34:55.822980 23976 ts_tablet_manager.cc:1918] T 263d652ffd924f3f939165f8fdb413c4 P a595b5e3641944aabba56c9680184959: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:34:55.823294 23976 log.cc:1196] T 263d652ffd924f3f939165f8fdb413c4 P a595b5e3641944aabba56c9680184959: Deleting WAL directory at /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786127579900523-11321-0/minicluster-data/ts-2/wal/wals/263d652ffd924f3f939165f8fdb413c4
I20260807 18:34:55.823618 11321 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkz56XN/build/debug/bin/kudu with pid 23639
I20260807 18:34:55.827903 23976 ts_tablet_manager.cc:1939] T 263d652ffd924f3f939165f8fdb413c4 P a595b5e3641944aabba56c9680184959: Deleting consensus metadata
I20260807 18:34:55.829663 23392 catalog_manager.cc:4841] TS a595b5e3641944aabba56c9680184959 (127.11.14.67:40005): tablet 263d652ffd924f3f939165f8fdb413c4 (table test-table [id=e930a37adec1430ca4253f5cbd68e183]) successfully deleted
I20260807 18:34:55.832500 23944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:34:55.832109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.65:45283 (local address 127.0.0.1:42340)
0807 18:34:55.832326 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0807 18:34:55.832404 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.65:45283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121}
W20260807 18:34:55.832832 23393 catalog_manager.cc:4584] TS 5a7c20f774ca4940b145dfb3bab0eeba (127.11.14.65:45283): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 263d652ffd924f3f939165f8fdb413c4: Network error: Client connection negotiation failed: client connection to 127.11.14.65:45283: connect: Connection refused (error 111)
W20260807 18:34:55.844389 23944 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:34:55.841708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.66:35901 (local address 127.0.0.1:45288)
0807 18:34:55.842263 (+ 555us) negotiation.cc:107] Waiting for socket to connect
0807 18:34:55.842280 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 18:34:55.842377 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:34:55.844265 (+ 1888us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.66:35901: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":456}
I20260807 18:34:55.845148 11321 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkz56XN/build/debug/bin/kudu with pid 23784
I20260807 18:34:55.853866 11321 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkz56XN/build/debug/bin/kudu with pid 23373
I20260807 18:34:55.868964 11321 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260807 18:34:56.210538 11321 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260807 18:34:56.210680 11321 mini_ranger.cc:66] Stopping Ranger...
I20260807 18:34:56.556190 11321 mini_ranger.cc:68] Stopped Ranger
2026-08-07 18:34:56.556 UTC [22927] LOG: received smart shutdown request
2026-08-07 18:34:56.560 UTC [22927] LOG: background worker "logical replication launcher" (PID 22936) exited with exit code 1
2026-08-07 18:34:56.560 UTC [22931] LOG: shutting down
2026-08-07 18:34:56.732 UTC [22927] LOG: database system is shut down
2026-08-07T18:34:56Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (49916 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:34:56 dist-test-slave-6k22 krb5kdc[23989](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:34:56 dist-test-slave-6k22 krb5kdc[23989](info): set up 2 sockets
Aug 07 18:34:56 dist-test-slave-6k22 krb5kdc[23989](info): commencing operation
krb5kdc: starting...
W20260807 18:34:57.953866 11321 mini_kdc.cc:121] Time spent starting KDC: real 1.071s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/krb5kdc/test-user.keytab.
Aug 07 18:34:58 dist-test-slave-6k22 krb5kdc[23989](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127698, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:34:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:34:58Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.11.14.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.11.14.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.11.14.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/krb5kdc/rangeradmin_127.11.14.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.11.14.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/krb5kdc/rangeradmin_127.11.14.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.11.14.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.11.14.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.11.14.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/krb5kdc/rangerlookup_127.11.14.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.11.14.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/krb5kdc/rangerlookup_127.11.14.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.11.14.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.11.14.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.11.14.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/krb5kdc/HTTP_127.11.14.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.11.14.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/krb5kdc/HTTP_127.11.14.84@KRBTEST.COM.keytab.
I20260807 18:34:58.131003 11321 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskkz56XN/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/minicluster-data/postgres -l logfile start
2026-08-07 18:35:00.695 UTC [24022] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-07 18:35:00.695 UTC [24022] LOG: listening on IPv4 address "127.11.14.84", port 57501
2026-08-07 18:35:00.699 UTC [24022] LOG: listening on Unix socket "/tmp/.s.PGSQL.57501"
2026-08-07 18:35:00.721 UTC [24025] LOG: database system was shut down at 2026-08-07 18:34:59 UTC
2026-08-07 18:35:00.725 UTC [24022] LOG: database system is ready to accept connections
I20260807 18:35:01.810109 11321 mini_postgres.cc:96] Postgres bound to 57501
2026-08-07 18:35:01.817 UTC [24037] FATAL: database "slave" does not exist
127.11.14.84:57501 - accepting connections
I20260807 18:35:01.817979 11321 mini_ranger.cc:161] Starting Ranger...
I20260807 18:35:01.833565 11321 mini_ranger.cc:84] Created miniranger Postgres user
I20260807 18:35:02.312579 11321 mini_ranger.cc:87] Created ranger Postgres database
I20260807 18:35:02.312667 11321 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/minicluster-data/ranger-admin
2026-08-07 18:35:02,597 [I] DB FLAVOR :POSTGRES
2026-08-07 18:35:02,597 [I] --------- Verifying Ranger DB connection ---------
2026-08-07 18:35:02,597 [I] Checking connection..
2026-08-07 18:35:02,597 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskkz56XN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskkz56XN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.14.84:57501/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-07 18:35:02,867 [I] Checking connection passed.
2026-08-07 18:35:02,867 [I] --------- Verifying version history table ---------
2026-08-07 18:35:02,867 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskkz56XN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskkz56XN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.14.84:57501/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 18:35:03,152 [I] Table x_db_version_h does not exist in database ranger
2026-08-07 18:35:03,152 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskkz56XN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskkz56XN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.14.84:57501/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 18:35:03,421 [I] Table x_db_version_h does not exist in database ranger
2026-08-07 18:35:03,421 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-07 18:35:03,421 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskkz56XN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskkz56XN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.14.84:57501/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskkz56XN/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-07 18:35:03.684 UTC [24106] WARNING: there is no transaction in progress
2026-08-07 18:35:03,691 [I] create_dbversion_catalog.sql file imported successfully
2026-08-07 18:35:03,692 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskkz56XN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskkz56XN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.14.84:57501/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 18:35:03,925 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-07 18:35:03,925 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-07 18:35:03,925 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskkz56XN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskkz56XN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.14.84:57501/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-07 18:35:04,198 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskkz56XN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskkz56XN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.14.84:57501/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-07 18:35:04,466 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskkz56XN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskkz56XN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.14.84:57501/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-6k22.c.gcp-upstream.internal','N') ;"
2026-08-07 18:35:04,721 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskkz56XN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskkz56XN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.14.84:57501/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-07 18:35:04,986 [I] Table x_portal_user does not exist in database ranger
2026-08-07 18:35:04,986 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskkz56XN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskkz56XN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.14.84:57501/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-07 18:35:05,253 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-07 18:35:05,254 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskkz56XN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskkz56XN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.14.84:57501/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-07 18:35:05,517 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskkz56XN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskkz56XN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.14.84:57501/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-07 18:35:05,782 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-07 18:35:05,782 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskkz56XN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskkz56XN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.14.84:57501/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskkz56XN/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-07 18:35:07.087 UTC [24225] WARNING: there is no transaction in progress
2026-08-07 18:35:07.103 UTC [24225] WARNING: there is no transaction in progress
2026-08-07 18:35:07.119 UTC [24225] WARNING: there is no transaction in progress
2026-08-07 18:35:07.135 UTC [24225] WARNING: there is no transaction in progress
2026-08-07 18:35:07.151 UTC [24225] WARNING: there is no transaction in progress
2026-08-07 18:35:07.168 UTC [24225] WARNING: there is no transaction in progress
2026-08-07 18:35:07.221 UTC [24225] WARNING: there is no transaction in progress
2026-08-07 18:35:07.227 UTC [24225] WARNING: there is no transaction in progress
2026-08-07 18:35:07.242 UTC [24225] WARNING: there is no transaction in progress
2026-08-07 18:35:07.251 UTC [24225] WARNING: there is no transaction in progress
2026-08-07 18:35:07.262 UTC [24225] WARNING: there is no transaction in progress
2026-08-07 18:35:07.275 UTC [24225] WARNING: there is no transaction in progress
2026-08-07 18:35:07.284 UTC [24225] WARNING: there is no transaction in progress
2026-08-07 18:35:07.291 UTC [24225] WARNING: there is no transaction in progress
2026-08-07 18:35:07.297 UTC [24225] WARNING: there is no transaction in progress
2026-08-07 18:35:07.304 UTC [24225] WARNING: there is no transaction in progress
2026-08-07 18:35:07,934 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-07 18:35:07,935 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskkz56XN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskkz56XN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.14.84:57501/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-07 18:35:08,191 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-07 18:35:08,192 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskkz56XN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskkz56XN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.14.84:57501/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-07 18:35:08,461 [I] Table x_portal_user already exists in database 'ranger'
2026-08-07 18:35:08,462 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskkz56XN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskkz56XN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.14.84:57501/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-07 18:35:08,728 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-07 18:35:08,729 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskkz56XN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskkz56XN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.14.84:57501/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-07 18:35:08,981 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskkz56XN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskkz56XN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.14.84:57501/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-07 18:35:09,234 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskkz56XN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskkz56XN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.14.84:57501/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-6k22.c.gcp-upstream.internal';"
2026-08-07 18:35:09,490 [I] CORE_DB_SCHEMA import status has been updated
2026-08-07 18:35:09,490 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskkz56XN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskkz56XN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.14.84:57501/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-07 18:35:09,752 [I] DB_PATCHES have already been applied
I20260807 18:35:09.758277 11321 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/minicluster-data/ranger-admin:/tmp/dist-test-taskkz56XN/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskkz56XN/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskkz56XN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskkz56XN/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260807 18:35:09.758363 11321 mini_ranger.cc:193] Using host: 127.11.14.84
I20260807 18:35:09.762861 11321 mini_ranger.cc:227] Ranger admin URL: http://127.11.14.84:51023
Aug 07, 2026 6:35:10 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 07, 2026 6:35:10 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 07, 2026 6:35:10 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/minicluster-data/ranger-admin/ews/webapp
Aug 07, 2026 6:35:10 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 07, 2026 6:35:10 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 07, 2026 6:35:10 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/minicluster-data/ranger-admin/ews/webapp].
Aug 07, 2026 6:35:10 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260807 18:35:10.959442 11321 mini_ranger.cc:160] Time spent starting Ranger: real 9.141s user 0.000s sys 0.011s
Aug 07, 2026 6:35:31 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 07, 2026 6:35:31 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.MetricsREST
Aug 07, 2026 6:35:31 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 07, 2026 6:35:31 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 07, 2026 6:35:31 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 07, 2026 6:35:31 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 07, 2026 6:35:31 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 07, 2026 6:35:31 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 07, 2026 6:35:31 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 07, 2026 6:35:31 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 07, 2026 6:35:31 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 07, 2026 6:35:31 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 07, 2026 6:35:31 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 07, 2026 6:35:31 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 07, 2026 6:35:31 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 07, 2026 6:35:31 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 07, 2026 6:35:31 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 07, 2026 6:35:32 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260807 18:35:32.679114 11321 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.11.14.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.11.14.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.11.14.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/krb5kdc/rangerkms_127.11.14.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.11.14.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/krb5kdc/rangerkms_127.11.14.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.11.14.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.11.14.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.11.14.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/krb5kdc/HTTP_127.11.14.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.11.14.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/krb5kdc/HTTP_127.11.14.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 07 18:35:32 dist-test-slave-6k22 krb5kdc[23989](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127732, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260807 18:35:32.772207 11321 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260807 18:35:32.782985 11321 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260807 18:35:33.680388 11321 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260807 18:35:33.680461 11321 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/minicluster-data/ranger-kms
I20260807 18:35:33.680475 11321 mini_ranger_kms.cc:227] Using postgres at 127.11.14.84:57501
2026-08-07 18:35:33,882 [I] DB FLAVOR :POSTGRES
2026-08-07 18:35:33,882 [I] --------- Verifying Ranger DB connection ---------
2026-08-07 18:35:33,882 [I] Checking connection
2026-08-07 18:35:33,883 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskkz56XN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.14.84:57501/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-07 18:35:34,130 [I] connection success
2026-08-07 18:35:34,131 [I] --------- Verifying Ranger DB tables ---------
2026-08-07 18:35:34,131 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-07 18:35:34,131 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskkz56XN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.14.84:57501/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-07 18:35:34,386 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-07 18:35:34,387 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-07 18:35:34,387 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-07 18:35:34,387 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskkz56XN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.14.84:57501/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-07 18:35:34,670 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260807 18:35:35.127518 11321 mini_ranger_kms.cc:310] Created kms service
I20260807 18:35:35.303797 11321 mini_ranger_kms.cc:326] Created kudu user
I20260807 18:35:35.351841 11321 mini_ranger_kms.cc:343] Created rangerkms user
Aug 07 18:35:35 dist-test-slave-6k22 krb5kdc[23989](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127732, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.11.14.84@KRBTEST.COM
I20260807 18:35:35.756716 11321 mini_ranger_kms.cc:384] Added ranger policy
I20260807 18:35:35.757022 11321 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/minicluster-data/ranger-kms:/tmp/dist-test-taskkz56XN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskkz56XN/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskkz56XN/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260807 18:35:35.757053 11321 mini_ranger_kms.cc:240] Using host: 127.11.14.84
I20260807 18:35:35.762440 11321 mini_ranger_kms.cc:276] Ranger KMS PID: 24423
I20260807 18:35:35.762539 11321 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.11.14.84:50947
Aug 07, 2026 6:35:37 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 07, 2026 6:35:37 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 07, 2026 6:35:37 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 07, 2026 6:35:37 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 07, 2026 6:35:37 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/minicluster-data/ranger-kms/ews/webapp].
Aug 07, 2026 6:35:37 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260807 18:35:38.231015 11321 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.459s user 0.002s sys 0.185s
I20260807 18:35:38.231166 11321 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260807 18:35:38.231266 11321 mini_ranger_kms.cc:401] 127.11.14.84:50947/kms/v1/keys
Aug 07 18:35:38 dist-test-slave-6k22 krb5kdc[23989](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786127738, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.11.14.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 07 18:35:38 dist-test-slave-6k22 krb5kdc[23989](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127738, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.11.14.84@KRBTEST.COM for HTTP/127.11.14.84@KRBTEST.COM
Aug 07 18:35:39 dist-test-slave-6k22 krb5kdc[23989](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127738, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.11.14.84@KRBTEST.COM for HTTP/127.11.14.84@KRBTEST.COM
Aug 07 18:35:41 dist-test-slave-6k22 krb5kdc[23989](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127741, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.11.14.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.14.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.14.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.126.keytab.
Entry for principal kudu/127.11.14.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.126.keytab.
I20260807 18:35:41.510617 11321 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkz56XN/build/debug/bin/kudu
/tmp/dist-test-taskkz56XN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.14.126:41689
--webserver_interface=127.11.14.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.126.keytab
--principal=kudu/127.11.14.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.14.84:36705
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.14.126:41689
--ranger_config_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.14.84:50947/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:35:41.632097 24477 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:35:41.632400 24477 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:35:41.632478 24477 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:35:41.636513 24477 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:35:41.636595 24477 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:35:41.636634 24477 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:35:41.636691 24477 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:35:41.636732 24477 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:35:41.642151 24477 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.14.84:36705
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/minicluster-data/master-0/wal
--ranger_kms_url=127.11.14.84:50947/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.11.14.126:41689
--ranger_config_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.126.keytab
--principal=kudu/127.11.14.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.14.126:41689
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.14.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24477
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:35:41.643220 24477 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:35:41.644343 24477 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 18:35:41.653481 24477 server_base.cc:1034] running on GCE node
W20260807 18:35:41.653571 24482 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:35:41.653718 24487 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:35:41.653726 24484 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:35:41.654543 24477 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:35:41.656024 24477 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:35:41.657234 24477 hybrid_clock.cc:648] HybridClock initialized: now 1786127741657197 us; error 45 us; skew 500 ppm
Aug 07 18:35:41 dist-test-slave-6k22 krb5kdc[23989](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127741, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:35:41.667054 24477 init.cc:377] Logged in from keytab as kudu/127.11.14.126@KRBTEST.COM (short username kudu)
I20260807 18:35:41.668351 24477 webserver.cc:460] Webserver started at http://127.11.14.126:39669/ using document root <none> and password file <none>
I20260807 18:35:41.669088 24477 fs_manager.cc:359] Metadata directory not provided
I20260807 18:35:41.669184 24477 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:35:41.669440 24477 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 18:35:41 dist-test-slave-6k22 krb5kdc[23989](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127741, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.126@KRBTEST.COM for HTTP/127.11.14.84@KRBTEST.COM
I20260807 18:35:41.718524 24477 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/minicluster-data/master-0/data/instance:
uuid: "83eae714e78e487d80c3e68e2db3e6d2"
format_stamp: "Formatted at 2026-08-07 18:35:41 on dist-test-slave-6k22"
server_key: "11d77bb888e959e77c9c45aec4ac3b08"
server_key_iv: "9283dc7b90b1556f3152422e7df2f6a8"
server_key_version: "kuduclusterkey@0"
I20260807 18:35:41.719190 24477 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/minicluster-data/master-0/wal/instance:
uuid: "83eae714e78e487d80c3e68e2db3e6d2"
format_stamp: "Formatted at 2026-08-07 18:35:41 on dist-test-slave-6k22"
server_key: "11d77bb888e959e77c9c45aec4ac3b08"
server_key_iv: "9283dc7b90b1556f3152422e7df2f6a8"
server_key_version: "kuduclusterkey@0"
I20260807 18:35:41.723392 24477 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260807 18:35:41.742729 24494 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:35:41.744237 24477 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:35:41.744392 24477 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/minicluster-data/master-0/data,/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/minicluster-data/master-0/wal
uuid: "83eae714e78e487d80c3e68e2db3e6d2"
format_stamp: "Formatted at 2026-08-07 18:35:41 on dist-test-slave-6k22"
server_key: "11d77bb888e959e77c9c45aec4ac3b08"
server_key_iv: "9283dc7b90b1556f3152422e7df2f6a8"
server_key_version: "kuduclusterkey@0"
I20260807 18:35:41.744529 24477 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:35:41.761373 24477 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:35:41.764361 24477 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:35:41.764566 24477 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:35:41.772794 24477 rpc_server.cc:307] RPC server started. Bound to: 127.11.14.126:41689
I20260807 18:35:41.772876 24551 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.14.126:41689 every 8 connection(s)
I20260807 18:35:41.773855 24477 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/minicluster-data/master-0/data/info.pb
I20260807 18:35:41.776871 24552 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:35:41.779225 11321 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkz56XN/build/debug/bin/kudu as pid 24477
I20260807 18:35:41.779385 11321 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/minicluster-data/master-0/wal/instance
I20260807 18:35:41.779732 11321 external_mini_cluster.cc:1428] Setting key 3bfd5192a2c373cd56b66f84ee861122
I20260807 18:35:41.783282 24552 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 83eae714e78e487d80c3e68e2db3e6d2: Bootstrap starting.
I20260807 18:35:41.785878 24552 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 83eae714e78e487d80c3e68e2db3e6d2: Neither blocks nor log segments found. Creating new log.
I20260807 18:35:41.786789 24552 log.cc:824] T 00000000000000000000000000000000 P 83eae714e78e487d80c3e68e2db3e6d2: Log is configured to *not* fsync() on all Append() calls
I20260807 18:35:41.789304 24552 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 83eae714e78e487d80c3e68e2db3e6d2: No bootstrap required, opened a new log
Aug 07 18:35:41 dist-test-slave-6k22 krb5kdc[23989](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127741, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.14.126@KRBTEST.COM
I20260807 18:35:41.794392 24552 raft_consensus.cc:348] T 00000000000000000000000000000000 P 83eae714e78e487d80c3e68e2db3e6d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83eae714e78e487d80c3e68e2db3e6d2" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 41689 } }
I20260807 18:35:41.794690 24552 raft_consensus.cc:374] T 00000000000000000000000000000000 P 83eae714e78e487d80c3e68e2db3e6d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:35:41.794799 24552 raft_consensus.cc:729] T 00000000000000000000000000000000 P 83eae714e78e487d80c3e68e2db3e6d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83eae714e78e487d80c3e68e2db3e6d2, State: Initialized, Role: FOLLOWER
I20260807 18:35:41.795323 24552 consensus_queue.cc:260] T 00000000000000000000000000000000 P 83eae714e78e487d80c3e68e2db3e6d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83eae714e78e487d80c3e68e2db3e6d2" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 41689 } }
I20260807 18:35:41.795506 24552 raft_consensus.cc:388] T 00000000000000000000000000000000 P 83eae714e78e487d80c3e68e2db3e6d2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:35:41.795631 24552 raft_consensus.cc:482] T 00000000000000000000000000000000 P 83eae714e78e487d80c3e68e2db3e6d2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:35:41.795774 24552 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 83eae714e78e487d80c3e68e2db3e6d2 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:35:41.796835 24552 raft_consensus.cc:504] T 00000000000000000000000000000000 P 83eae714e78e487d80c3e68e2db3e6d2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83eae714e78e487d80c3e68e2db3e6d2" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 41689 } }
I20260807 18:35:41.797207 24552 leader_election.cc:302] T 00000000000000000000000000000000 P 83eae714e78e487d80c3e68e2db3e6d2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 83eae714e78e487d80c3e68e2db3e6d2; no voters:
I20260807 18:35:41.797600 24552 leader_election.cc:288] T 00000000000000000000000000000000 P 83eae714e78e487d80c3e68e2db3e6d2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:35:41.797670 24559 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 83eae714e78e487d80c3e68e2db3e6d2 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:35:41.797829 24557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:35:41.781279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39164 (local address 127.11.14.126:41689)
0807 18:35:41.781780 (+ 501us) server_negotiation.cc:206] Beginning negotiation
0807 18:35:41.781802 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0807 18:35:41.781857 (+ 55us) server_negotiation.cc:407] Connection header received
0807 18:35:41.782807 (+ 950us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:35:41.782835 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:35:41.783306 (+ 471us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:35:41.783729 (+ 423us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:35:41.784595 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:35:41.786224 (+ 1629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:35:41.787500 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:35:41.788181 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:35:41.788337 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:35:41.791810 (+ 3473us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:35:41.791817 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:35:41.791846 (+ 29us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:35:41.791877 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:35:41.793937 (+ 2060us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:35:41.794459 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:35:41.794465 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:35:41.794471 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:35:41.794596 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:35:41.794947 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:35:41.794951 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:35:41.794952 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:35:41.795448 (+ 496us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:35:41.795669 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0807 18:35:41.796475 (+ 806us) server_negotiation.cc:299] Negotiation successful
0807 18:35:41.796770 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":335,"thread_start_us":128,"threads_started":1}
I20260807 18:35:41.798133 24559 raft_consensus.cc:686] T 00000000000000000000000000000000 P 83eae714e78e487d80c3e68e2db3e6d2 [term 1 LEADER]: Becoming Leader. State: Replica: 83eae714e78e487d80c3e68e2db3e6d2, State: Running, Role: LEADER
I20260807 18:35:41.798512 24559 consensus_queue.cc:237] T 00000000000000000000000000000000 P 83eae714e78e487d80c3e68e2db3e6d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83eae714e78e487d80c3e68e2db3e6d2" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 41689 } }
I20260807 18:35:41.799028 24552 sys_catalog.cc:564] T 00000000000000000000000000000000 P 83eae714e78e487d80c3e68e2db3e6d2 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:35:41.801339 24561 sys_catalog.cc:455] T 00000000000000000000000000000000 P 83eae714e78e487d80c3e68e2db3e6d2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 83eae714e78e487d80c3e68e2db3e6d2. Latest consensus state: current_term: 1 leader_uuid: "83eae714e78e487d80c3e68e2db3e6d2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83eae714e78e487d80c3e68e2db3e6d2" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 41689 } } }
I20260807 18:35:41.801450 24552 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260807 18:35:41.801800 24562 sys_catalog.cc:455] T 00000000000000000000000000000000 P 83eae714e78e487d80c3e68e2db3e6d2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "83eae714e78e487d80c3e68e2db3e6d2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83eae714e78e487d80c3e68e2db3e6d2" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 41689 } } }
I20260807 18:35:41.801972 24562 sys_catalog.cc:458] T 00000000000000000000000000000000 P 83eae714e78e487d80c3e68e2db3e6d2 [sys.catalog]: This master's current role is: LEADER
I20260807 18:35:41.801497 24561 sys_catalog.cc:458] T 00000000000000000000000000000000 P 83eae714e78e487d80c3e68e2db3e6d2 [sys.catalog]: This master's current role is: LEADER
I20260807 18:35:41.805176 24575 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:35:41.810297 24575 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:35:41.817591 24575 catalog_manager.cc:1347] Generated new cluster ID: 5c3a69c331b94744bd09d5d5e60d7540
I20260807 18:35:41.817734 24575 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:35:41.836035 24575 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:35:41.837541 24575 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:35:41.848310 24575 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 83eae714e78e487d80c3e68e2db3e6d2: Generated new TSK 0
I20260807 18:35:41.849462 24575 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.14.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.14.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.14.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.65.keytab.
Entry for principal kudu/127.11.14.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.65.keytab.
I20260807 18:35:42.626682 11321 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkz56XN/build/debug/bin/kudu
/tmp/dist-test-taskkz56XN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.14.84:50947/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.14.65:0
--local_ip_for_outbound_sockets=127.11.14.65
--webserver_interface=127.11.14.65
--webserver_port=0
--tserver_master_addrs=127.11.14.126:41689
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.65.keytab
--principal=kudu/127.11.14.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.14.84:36705
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:35:42.814668 24603 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:35:42.814975 24603 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:35:42.815106 24603 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:35:42.819746 24603 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:35:42.819839 24603 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:35:42.819960 24603 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.14.65
I20260807 18:35:42.827418 24603 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.14.84:36705
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.11.14.84:50947/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.65.keytab
--principal=kudu/127.11.14.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.14.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.14.65
--webserver_port=0
--tserver_master_addrs=127.11.14.126:41689
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24603
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.14.65
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:35:42.828847 24603 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:35:42.830047 24603 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:35:42.842263 24612 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:35:42.842307 24610 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:35:42.842935 24609 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:35:42.843063 24603 server_base.cc:1034] running on GCE node
I20260807 18:35:42.843489 24603 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:35:42.844273 24603 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:35:42.845497 24603 hybrid_clock.cc:648] HybridClock initialized: now 1786127742845420 us; error 89 us; skew 500 ppm
Aug 07 18:35:42 dist-test-slave-6k22 krb5kdc[23989](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127742, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:35:42.850240 24603 init.cc:377] Logged in from keytab as kudu/127.11.14.65@KRBTEST.COM (short username kudu)
I20260807 18:35:42.851459 24603 webserver.cc:460] Webserver started at http://127.11.14.65:33697/ using document root <none> and password file <none>
I20260807 18:35:42.852274 24603 fs_manager.cc:359] Metadata directory not provided
I20260807 18:35:42.852432 24603 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:35:42.852793 24603 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 18:35:42 dist-test-slave-6k22 krb5kdc[23989](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127742, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.65@KRBTEST.COM for HTTP/127.11.14.84@KRBTEST.COM
I20260807 18:35:42.877900 24603 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/minicluster-data/ts-0/data/instance:
uuid: "035b879bf04848f3a6c460fb1cb72d52"
format_stamp: "Formatted at 2026-08-07 18:35:42 on dist-test-slave-6k22"
server_key: "708119013fc52ad2e854204f348193fc"
server_key_iv: "927a417be604ce85b4365414960b2b22"
server_key_version: "kuduclusterkey@0"
I20260807 18:35:42.878772 24603 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/minicluster-data/ts-0/wal/instance:
uuid: "035b879bf04848f3a6c460fb1cb72d52"
format_stamp: "Formatted at 2026-08-07 18:35:42 on dist-test-slave-6k22"
server_key: "708119013fc52ad2e854204f348193fc"
server_key_iv: "927a417be604ce85b4365414960b2b22"
server_key_version: "kuduclusterkey@0"
I20260807 18:35:42.885267 24603 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260807 18:35:42.907809 24622 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:35:42.912431 24603 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.003s sys 0.000s
I20260807 18:35:42.912600 24603 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/minicluster-data/ts-0/data,/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/minicluster-data/ts-0/wal
uuid: "035b879bf04848f3a6c460fb1cb72d52"
format_stamp: "Formatted at 2026-08-07 18:35:42 on dist-test-slave-6k22"
server_key: "708119013fc52ad2e854204f348193fc"
server_key_iv: "927a417be604ce85b4365414960b2b22"
server_key_version: "kuduclusterkey@0"
I20260807 18:35:42.912771 24603 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:35:42.952601 24603 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:35:42.956683 24603 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:35:42.956950 24603 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:35:42.957959 24603 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:35:42.959170 24603 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:35:42.959280 24603 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:35:42.959406 24603 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:35:42.959489 24603 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:35:42.974965 24603 rpc_server.cc:307] RPC server started. Bound to: 127.11.14.65:37667
I20260807 18:35:42.976363 24603 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/minicluster-data/ts-0/data/info.pb
I20260807 18:35:42.978878 24747 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.14.65:37667 every 8 connection(s)
I20260807 18:35:42.982393 11321 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkz56XN/build/debug/bin/kudu as pid 24603
I20260807 18:35:42.982518 11321 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/minicluster-data/ts-0/wal/instance
I20260807 18:35:42.982880 11321 external_mini_cluster.cc:1428] Setting key 5aab332b15ef00f8c27e0a651eabb9d6
Aug 07 18:35:42 dist-test-slave-6k22 krb5kdc[23989](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127742, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.65@KRBTEST.COM for kudu/127.11.14.126@KRBTEST.COM
I20260807 18:35:43.002606 24750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:35:42.980871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.65:39271 (local address 127.11.14.126:41689)
0807 18:35:42.981126 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0807 18:35:42.981131 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:35:42.982102 (+ 971us) server_negotiation.cc:407] Connection header received
0807 18:35:42.988194 (+ 6092us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:35:42.988198 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:35:42.988256 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:35:42.988319 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:35:42.990081 (+ 1762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:35:42.991059 (+ 978us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:35:42.992862 (+ 1803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:35:42.993248 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:35:42.993322 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:35:42.997973 (+ 4651us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:35:42.997977 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:35:42.997980 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:35:42.998007 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:35:43.000034 (+ 2027us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:35:43.000739 (+ 705us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:35:43.000743 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:35:43.000744 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:35:43.000835 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:35:43.001235 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:35:43.001238 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:35:43.001240 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:35:43.001480 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:35:43.001563 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0807 18:35:43.002246 (+ 683us) server_negotiation.cc:299] Negotiation successful
0807 18:35:43.002385 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":176,"thread_start_us":72,"threads_started":1}
I20260807 18:35:43.004724 24751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:35:42.981361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:41689 (local address 127.11.14.65:39271)
0807 18:35:42.981926 (+ 565us) negotiation.cc:107] Waiting for socket to connect
0807 18:35:42.981980 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0807 18:35:42.987932 (+ 5952us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:35:42.988774 (+ 842us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:35:42.988784 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:35:42.989189 (+ 405us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:35:42.989914 (+ 725us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:35:42.989933 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:35:42.991226 (+ 1293us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:35:42.991229 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:35:42.992688 (+ 1459us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:35:42.992699 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:35:42.993746 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:35:42.993751 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:35:42.993966 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:35:42.995742 (+ 1776us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:35:42.995763 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:35:42.997760 (+ 1997us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:35:43.000191 (+ 2431us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:35:43.000198 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:35:43.000214 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:35:43.000606 (+ 392us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:35:43.000946 (+ 340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:35:43.000949 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:35:43.000951 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:35:43.001124 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:35:43.001642 (+ 518us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:35:43.001649 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:35:43.001959 (+ 310us) client_negotiation.cc:769] Sending connection context
0807 18:35:43.003089 (+ 1130us) client_negotiation.cc:240] Negotiation successful
0807 18:35:43.003432 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":337,"thread_start_us":163,"threads_started":1}
I20260807 18:35:43.006645 24748 heartbeater.cc:347] Connected to a master server at 127.11.14.126:41689
I20260807 18:35:43.006928 24748 heartbeater.cc:464] Registering TS with master...
I20260807 18:35:43.007675 24748 heartbeater.cc:511] Master 127.11.14.126:41689 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.11.14.66@KRBTEST.COM; defaulting to no policy
I20260807 18:35:43.009796 24515 ts_manager.cc:194] Registered new tserver with Master: 035b879bf04848f3a6c460fb1cb72d52 (127.11.14.65:37667)
I20260807 18:35:43.011411 24515 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.14.65@KRBTEST.COM'} at 127.11.14.65:39271
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.14.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.14.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.66.keytab.
Entry for principal kudu/127.11.14.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.66.keytab.
I20260807 18:35:43.035722 11321 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkz56XN/build/debug/bin/kudu
/tmp/dist-test-taskkz56XN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.14.84:50947/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.14.66:0
--local_ip_for_outbound_sockets=127.11.14.66
--webserver_interface=127.11.14.66
--webserver_port=0
--tserver_master_addrs=127.11.14.126:41689
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.66.keytab
--principal=kudu/127.11.14.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.14.84:36705
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 18:35:43 dist-test-slave-6k22 krb5kdc[23989](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786127743, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260807 18:35:43.210474 24756 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:35:43.210951 24756 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:35:43.211083 24756 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:35:43.217548 24756 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:35:43.217701 24756 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:35:43.217844 24756 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.14.66
I20260807 18:35:43.225941 24756 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.14.84:36705
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.11.14.84:50947/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.66.keytab
--principal=kudu/127.11.14.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.14.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.14.66
--webserver_port=0
--tserver_master_addrs=127.11.14.126:41689
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24756
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.14.66
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:35:43.227391 24756 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:35:43.228820 24756 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:35:43.246295 24763 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:35:43.247956 24766 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:35:43.248272 24764 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:35:43.248703 24756 server_base.cc:1034] running on GCE node
I20260807 18:35:43.249476 24756 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:35:43.250334 24756 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:35:43.251536 24756 hybrid_clock.cc:648] HybridClock initialized: now 1786127743251500 us; error 50 us; skew 500 ppm
Aug 07 18:35:43 dist-test-slave-6k22 krb5kdc[23989](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127743, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:35:43.255689 24756 init.cc:377] Logged in from keytab as kudu/127.11.14.66@KRBTEST.COM (short username kudu)
I20260807 18:35:43.256932 24756 webserver.cc:460] Webserver started at http://127.11.14.66:46525/ using document root <none> and password file <none>
I20260807 18:35:43.257714 24756 fs_manager.cc:359] Metadata directory not provided
I20260807 18:35:43.257812 24756 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:35:43.258149 24756 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 18:35:43 dist-test-slave-6k22 krb5kdc[23989](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127743, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.66@KRBTEST.COM for HTTP/127.11.14.84@KRBTEST.COM
I20260807 18:35:43.275250 24756 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/minicluster-data/ts-1/data/instance:
uuid: "1a1fb08cffbd4f619e7354cd653b0157"
format_stamp: "Formatted at 2026-08-07 18:35:43 on dist-test-slave-6k22"
server_key: "d0afd73858240786b5322de26bfae902"
server_key_iv: "e3267af7d87fc8afbb285244cbbb20a7"
server_key_version: "kuduclusterkey@0"
I20260807 18:35:43.276111 24756 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/minicluster-data/ts-1/wal/instance:
uuid: "1a1fb08cffbd4f619e7354cd653b0157"
format_stamp: "Formatted at 2026-08-07 18:35:43 on dist-test-slave-6k22"
server_key: "d0afd73858240786b5322de26bfae902"
server_key_iv: "e3267af7d87fc8afbb285244cbbb20a7"
server_key_version: "kuduclusterkey@0"
I20260807 18:35:43.281586 24756 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 18:35:43.312073 24773 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:35:43.313719 24756 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260807 18:35:43.313884 24756 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/minicluster-data/ts-1/data,/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/minicluster-data/ts-1/wal
uuid: "1a1fb08cffbd4f619e7354cd653b0157"
format_stamp: "Formatted at 2026-08-07 18:35:43 on dist-test-slave-6k22"
server_key: "d0afd73858240786b5322de26bfae902"
server_key_iv: "e3267af7d87fc8afbb285244cbbb20a7"
server_key_version: "kuduclusterkey@0"
I20260807 18:35:43.314044 24756 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:35:43.345850 24756 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:35:43.350232 24756 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:35:43.350459 24756 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:35:43.351202 24756 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:35:43.352298 24756 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:35:43.352370 24756 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:35:43.352466 24756 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:35:43.352527 24756 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:35:43.366561 24756 rpc_server.cc:307] RPC server started. Bound to: 127.11.14.66:39151
I20260807 18:35:43.366559 24896 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.14.66:39151 every 8 connection(s)
I20260807 18:35:43.368093 24756 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/minicluster-data/ts-1/data/info.pb
I20260807 18:35:43.375813 11321 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkz56XN/build/debug/bin/kudu as pid 24756
I20260807 18:35:43.375993 11321 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/minicluster-data/ts-1/wal/instance
I20260807 18:35:43.376303 11321 external_mini_cluster.cc:1428] Setting key fa85fd12720e2dac9f1807c841d0c328
Aug 07 18:35:43 dist-test-slave-6k22 krb5kdc[23989](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127743, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.66@KRBTEST.COM for kudu/127.11.14.126@KRBTEST.COM
I20260807 18:35:43.384881 24750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:35:43.370194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.66:54897 (local address 127.11.14.126:41689)
0807 18:35:43.370367 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0807 18:35:43.370373 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:35:43.371364 (+ 991us) server_negotiation.cc:407] Connection header received
0807 18:35:43.372992 (+ 1628us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:35:43.372995 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:35:43.373045 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:35:43.373126 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:35:43.374867 (+ 1741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:35:43.375421 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:35:43.377205 (+ 1784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:35:43.377517 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:35:43.377644 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:35:43.380589 (+ 2945us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:35:43.380595 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:35:43.380600 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:35:43.380634 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:35:43.382622 (+ 1988us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:35:43.383169 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:35:43.383172 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:35:43.383174 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:35:43.383262 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:35:43.383642 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:35:43.383646 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:35:43.383647 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:35:43.383844 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:35:43.383959 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0807 18:35:43.384521 (+ 562us) server_negotiation.cc:299] Negotiation successful
0807 18:35:43.384654 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260807 18:35:43.385850 24899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:35:43.370585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:41689 (local address 127.11.14.66:54897)
0807 18:35:43.371159 (+ 574us) negotiation.cc:107] Waiting for socket to connect
0807 18:35:43.371230 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0807 18:35:43.372688 (+ 1458us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:35:43.373441 (+ 753us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:35:43.373451 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:35:43.373927 (+ 476us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:35:43.374689 (+ 762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:35:43.374711 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:35:43.375596 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:35:43.375600 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:35:43.377079 (+ 1479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:35:43.377093 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:35:43.377842 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:35:43.377847 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:35:43.378043 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:35:43.378718 (+ 675us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:35:43.378736 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:35:43.380318 (+ 1582us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:35:43.382743 (+ 2425us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:35:43.382750 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:35:43.382767 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:35:43.383050 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:35:43.383363 (+ 313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:35:43.383366 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:35:43.383368 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:35:43.383519 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:35:43.383959 (+ 440us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:35:43.383967 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:35:43.384259 (+ 292us) client_negotiation.cc:769] Sending connection context
0807 18:35:43.384462 (+ 203us) client_negotiation.cc:240] Negotiation successful
0807 18:35:43.384845 (+ 383us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":300,"thread_start_us":128,"threads_started":1}
I20260807 18:35:43.387101 24897 heartbeater.cc:347] Connected to a master server at 127.11.14.126:41689
I20260807 18:35:43.387382 24897 heartbeater.cc:464] Registering TS with master...
I20260807 18:35:43.387977 24897 heartbeater.cc:511] Master 127.11.14.126:41689 requested a full tablet report, sending...
I20260807 18:35:43.389086 24515 ts_manager.cc:194] Registered new tserver with Master: 1a1fb08cffbd4f619e7354cd653b0157 (127.11.14.66:39151)
I20260807 18:35:43.389659 24515 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.14.66@KRBTEST.COM'} at 127.11.14.66:54897
WARNING: no policy specified for kudu/127.11.14.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.14.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.14.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.67.keytab.
Entry for principal kudu/127.11.14.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.67.keytab.
I20260807 18:35:43.420044 11321 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkz56XN/build/debug/bin/kudu
/tmp/dist-test-taskkz56XN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.14.84:50947/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.14.67:0
--local_ip_for_outbound_sockets=127.11.14.67
--webserver_interface=127.11.14.67
--webserver_port=0
--tserver_master_addrs=127.11.14.126:41689
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.67.keytab
--principal=kudu/127.11.14.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.14.84:36705
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 18:35:43 dist-test-slave-6k22 krb5kdc[23989](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127743, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.126@KRBTEST.COM for HTTP/127.11.14.84@KRBTEST.COM
W20260807 18:35:43.603957 24902 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:35:43.604271 24902 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:35:43.604403 24902 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:35:43.608695 24902 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:35:43.608794 24902 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:35:43.608915 24902 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.14.67
I20260807 18:35:43.616803 24902 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.14.84:36705
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.11.14.84:50947/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.67.keytab
--principal=kudu/127.11.14.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.14.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.14.67
--webserver_port=0
--tserver_master_addrs=127.11.14.126:41689
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24902
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.14.67
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:35:43.618268 24902 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:35:43.619728 24902 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:35:43.633006 24907 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:35:43.633139 24912 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:35:43.634295 24902 server_base.cc:1034] running on GCE node
W20260807 18:35:43.636965 24909 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:35:43.637965 24902 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:35:43.638913 24902 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:35:43.641222 24902 hybrid_clock.cc:648] HybridClock initialized: now 1786127743641145 us; error 91 us; skew 500 ppm
Aug 07 18:35:43 dist-test-slave-6k22 krb5kdc[23989](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127743, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:35:43.644925 24902 init.cc:377] Logged in from keytab as kudu/127.11.14.67@KRBTEST.COM (short username kudu)
I20260807 18:35:43.646169 24902 webserver.cc:460] Webserver started at http://127.11.14.67:43441/ using document root <none> and password file <none>
I20260807 18:35:43.647001 24902 fs_manager.cc:359] Metadata directory not provided
I20260807 18:35:43.647104 24902 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:35:43.647387 24902 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 18:35:43 dist-test-slave-6k22 krb5kdc[23989](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127743, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.67@KRBTEST.COM for HTTP/127.11.14.84@KRBTEST.COM
I20260807 18:35:43.665088 24902 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/minicluster-data/ts-2/data/instance:
uuid: "82c9bc9e8d564f2f963df6aca6e26281"
format_stamp: "Formatted at 2026-08-07 18:35:43 on dist-test-slave-6k22"
server_key: "49cd4511ad56e814e9130b8df790ea6f"
server_key_iv: "1e73e6b6d4d3ba9cf1307ca857919997"
server_key_version: "kuduclusterkey@0"
I20260807 18:35:43.666039 24902 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/minicluster-data/ts-2/wal/instance:
uuid: "82c9bc9e8d564f2f963df6aca6e26281"
format_stamp: "Formatted at 2026-08-07 18:35:43 on dist-test-slave-6k22"
server_key: "49cd4511ad56e814e9130b8df790ea6f"
server_key_iv: "1e73e6b6d4d3ba9cf1307ca857919997"
server_key_version: "kuduclusterkey@0"
I20260807 18:35:43.686933 24902 fs_manager.cc:691] Time spent creating directory manager: real 0.020s user 0.000s sys 0.004s
I20260807 18:35:43.703522 24919 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:35:43.705276 24902 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260807 18:35:43.705440 24902 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/minicluster-data/ts-2/data,/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/minicluster-data/ts-2/wal
uuid: "82c9bc9e8d564f2f963df6aca6e26281"
format_stamp: "Formatted at 2026-08-07 18:35:43 on dist-test-slave-6k22"
server_key: "49cd4511ad56e814e9130b8df790ea6f"
server_key_iv: "1e73e6b6d4d3ba9cf1307ca857919997"
server_key_version: "kuduclusterkey@0"
I20260807 18:35:43.705575 24902 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:35:43.750445 24902 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:35:43.754731 24902 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:35:43.755005 24902 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:35:43.755825 24902 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:35:43.756985 24902 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:35:43.757095 24902 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:35:43.757190 24902 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:35:43.757253 24902 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:35:43.769250 24902 rpc_server.cc:307] RPC server started. Bound to: 127.11.14.67:33565
I20260807 18:35:43.770498 24902 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/minicluster-data/ts-2/data/info.pb
I20260807 18:35:43.776844 11321 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkz56XN/build/debug/bin/kudu as pid 24902
I20260807 18:35:43.777005 11321 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/minicluster-data/ts-2/wal/instance
I20260807 18:35:43.777386 11321 external_mini_cluster.cc:1428] Setting key 63e76f3b877cc23ec33921a7ddbac045
I20260807 18:35:43.784920 25041 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.14.67:33565 every 8 connection(s)
Aug 07 18:35:43 dist-test-slave-6k22 krb5kdc[23989](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127743, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.67@KRBTEST.COM for kudu/127.11.14.126@KRBTEST.COM
I20260807 18:35:43.802222 24750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:35:43.788871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.67:55873 (local address 127.11.14.126:41689)
0807 18:35:43.789022 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0807 18:35:43.789028 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:35:43.789542 (+ 514us) server_negotiation.cc:407] Connection header received
0807 18:35:43.790849 (+ 1307us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:35:43.790853 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:35:43.790906 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:35:43.790954 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:35:43.792641 (+ 1687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:35:43.793418 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:35:43.795009 (+ 1591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:35:43.795330 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:35:43.795400 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:35:43.798275 (+ 2875us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:35:43.798279 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:35:43.798282 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:35:43.798307 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:35:43.799991 (+ 1684us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:35:43.800570 (+ 579us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:35:43.800573 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:35:43.800575 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:35:43.800653 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:35:43.801028 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:35:43.801031 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:35:43.801033 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:35:43.801246 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:35:43.801344 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0807 18:35:43.801953 (+ 609us) server_negotiation.cc:299] Negotiation successful
0807 18:35:43.802066 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260807 18:35:43.803833 25044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:35:43.787682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:41689 (local address 127.11.14.67:55873)
0807 18:35:43.789389 (+ 1707us) negotiation.cc:107] Waiting for socket to connect
0807 18:35:43.789442 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0807 18:35:43.790602 (+ 1160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:35:43.791372 (+ 770us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:35:43.791381 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:35:43.791810 (+ 429us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:35:43.792485 (+ 675us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:35:43.792502 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:35:43.793567 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:35:43.793570 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:35:43.794859 (+ 1289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:35:43.794871 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:35:43.795610 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:35:43.795615 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:35:43.795799 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:35:43.796472 (+ 673us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:35:43.796493 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:35:43.798092 (+ 1599us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:35:43.800126 (+ 2034us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:35:43.800132 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:35:43.800148 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:35:43.800461 (+ 313us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:35:43.800748 (+ 287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:35:43.800750 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:35:43.800752 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:35:43.800911 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:35:43.801403 (+ 492us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:35:43.801410 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:35:43.801697 (+ 287us) client_negotiation.cc:769] Sending connection context
0807 18:35:43.802394 (+ 697us) client_negotiation.cc:240] Negotiation successful
0807 18:35:43.802744 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1464,"thread_start_us":167,"threads_started":1}
I20260807 18:35:43.805168 25042 heartbeater.cc:347] Connected to a master server at 127.11.14.126:41689
I20260807 18:35:43.805414 25042 heartbeater.cc:464] Registering TS with master...
I20260807 18:35:43.805965 25042 heartbeater.cc:511] Master 127.11.14.126:41689 requested a full tablet report, sending...
I20260807 18:35:43.807056 24515 ts_manager.cc:194] Registered new tserver with Master: 82c9bc9e8d564f2f963df6aca6e26281 (127.11.14.67:33565)
I20260807 18:35:43.807848 24515 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.14.67@KRBTEST.COM'} at 127.11.14.67:55873
I20260807 18:35:43.816076 11321 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 18:35:43.816857 11321 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkz56XN/build/debug/bin/kudu with pid 24603
I20260807 18:35:43.858842 11321 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkz56XN/build/debug/bin/kudu with pid 24756
I20260807 18:35:43.898272 11321 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkz56XN/build/debug/bin/kudu with pid 24902
I20260807 18:35:43.915436 11321 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkz56XN/build/debug/bin/kudu with pid 24477
I20260807 18:35:43.928633 11321 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkz56XN/build/debug/bin/kudu
/tmp/dist-test-taskkz56XN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.14.126:41689
--webserver_interface=127.11.14.126
--webserver_port=39669
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.126.keytab
--principal=kudu/127.11.14.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.14.84:36705
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.14.126:41689
--ranger_config_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.14.84:50947/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.11.14.84:50947/kms with env {KRB5CCNAME=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:35:44.051103 25045 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:35:44.051375 25045 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:35:44.051419 25045 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:35:44.055392 25045 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:35:44.055459 25045 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:35:44.055481 25045 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:35:44.055501 25045 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:35:44.055519 25045 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:35:44.060953 25045 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.14.84:36705
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.11.14.84:50947/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.11.14.126:41689
--ranger_config_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.126.keytab
--principal=kudu/127.11.14.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.14.126:41689
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.14.126
--webserver_port=39669
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25045
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:35:44.061962 25045 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:35:44.062950 25045 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:35:44.071720 25051 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:35:44.071732 25053 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:35:44.071870 25045 server_base.cc:1034] running on GCE node
W20260807 18:35:44.072108 25055 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:35:44.072657 25045 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:35:44.073773 25045 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:35:44.074934 25045 hybrid_clock.cc:648] HybridClock initialized: now 1786127744074903 us; error 43 us; skew 500 ppm
Aug 07 18:35:44 dist-test-slave-6k22 krb5kdc[23989](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127744, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:35:44.077808 25045 init.cc:377] Logged in from keytab as kudu/127.11.14.126@KRBTEST.COM (short username kudu)
I20260807 18:35:44.078783 25045 webserver.cc:460] Webserver started at http://127.11.14.126:39669/ using document root <none> and password file <none>
I20260807 18:35:44.079336 25045 fs_manager.cc:359] Metadata directory not provided
I20260807 18:35:44.079437 25045 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 07 18:35:44 dist-test-slave-6k22 krb5kdc[23989](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127744, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.126@KRBTEST.COM for HTTP/127.11.14.84@KRBTEST.COM
I20260807 18:35:44.172235 25045 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260807 18:35:44.174649 25063 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:35:44.176491 25045 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 18:35:44.176651 25045 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/minicluster-data/master-0/data,/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/minicluster-data/master-0/wal
uuid: "83eae714e78e487d80c3e68e2db3e6d2"
format_stamp: "Formatted at 2026-08-07 18:35:41 on dist-test-slave-6k22"
server_key: "11d77bb888e959e77c9c45aec4ac3b08"
server_key_iv: "9283dc7b90b1556f3152422e7df2f6a8"
server_key_version: "kuduclusterkey@0"
I20260807 18:35:44.177433 25045 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:35:44.211143 25045 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:35:44.214239 25045 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:35:44.214398 25045 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:35:44.221768 25045 rpc_server.cc:307] RPC server started. Bound to: 127.11.14.126:41689
I20260807 18:35:44.221802 25120 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.14.126:41689 every 8 connection(s)
I20260807 18:35:44.222846 25045 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/minicluster-data/master-0/data/info.pb
I20260807 18:35:44.227174 25121 sys_catalog.cc:263] Verifying existing consensus state
I20260807 18:35:44.228858 25121 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 83eae714e78e487d80c3e68e2db3e6d2: Bootstrap starting.
I20260807 18:35:44.229233 11321 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkz56XN/build/debug/bin/kudu as pid 25045
I20260807 18:35:44.229701 11321 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkz56XN/build/debug/bin/kudu
/tmp/dist-test-taskkz56XN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.14.84:50947/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.14.65:37667
--local_ip_for_outbound_sockets=127.11.14.65
--tserver_master_addrs=127.11.14.126:41689
--webserver_port=33697
--webserver_interface=127.11.14.65
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.65.keytab
--principal=kudu/127.11.14.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.14.84:36705
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.11.14.84:50947/kms with env {KRB5CCNAME=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 18:35:44.245684 25121 log.cc:824] T 00000000000000000000000000000000 P 83eae714e78e487d80c3e68e2db3e6d2: Log is configured to *not* fsync() on all Append() calls
I20260807 18:35:44.252691 25121 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 83eae714e78e487d80c3e68e2db3e6d2: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 18:35:44.253113 25121 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 83eae714e78e487d80c3e68e2db3e6d2: Bootstrap complete.
I20260807 18:35:44.257501 25121 raft_consensus.cc:348] T 00000000000000000000000000000000 P 83eae714e78e487d80c3e68e2db3e6d2 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83eae714e78e487d80c3e68e2db3e6d2" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 41689 } }
I20260807 18:35:44.257926 25121 raft_consensus.cc:729] T 00000000000000000000000000000000 P 83eae714e78e487d80c3e68e2db3e6d2 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83eae714e78e487d80c3e68e2db3e6d2, State: Initialized, Role: FOLLOWER
I20260807 18:35:44.258490 25121 consensus_queue.cc:260] T 00000000000000000000000000000000 P 83eae714e78e487d80c3e68e2db3e6d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83eae714e78e487d80c3e68e2db3e6d2" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 41689 } }
I20260807 18:35:44.258642 25121 raft_consensus.cc:388] T 00000000000000000000000000000000 P 83eae714e78e487d80c3e68e2db3e6d2 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:35:44.258734 25121 raft_consensus.cc:482] T 00000000000000000000000000000000 P 83eae714e78e487d80c3e68e2db3e6d2 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:35:44.258862 25121 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 83eae714e78e487d80c3e68e2db3e6d2 [term 1 FOLLOWER]: Advancing to term 2
I20260807 18:35:44.260831 25121 raft_consensus.cc:504] T 00000000000000000000000000000000 P 83eae714e78e487d80c3e68e2db3e6d2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83eae714e78e487d80c3e68e2db3e6d2" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 41689 } }
I20260807 18:35:44.261181 25121 leader_election.cc:302] T 00000000000000000000000000000000 P 83eae714e78e487d80c3e68e2db3e6d2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 83eae714e78e487d80c3e68e2db3e6d2; no voters:
I20260807 18:35:44.261539 25121 leader_election.cc:288] T 00000000000000000000000000000000 P 83eae714e78e487d80c3e68e2db3e6d2 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260807 18:35:44.261677 25125 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 83eae714e78e487d80c3e68e2db3e6d2 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 18:35:44.261933 25125 raft_consensus.cc:686] T 00000000000000000000000000000000 P 83eae714e78e487d80c3e68e2db3e6d2 [term 2 LEADER]: Becoming Leader. State: Replica: 83eae714e78e487d80c3e68e2db3e6d2, State: Running, Role: LEADER
I20260807 18:35:44.262353 25125 consensus_queue.cc:237] T 00000000000000000000000000000000 P 83eae714e78e487d80c3e68e2db3e6d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83eae714e78e487d80c3e68e2db3e6d2" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 41689 } }
I20260807 18:35:44.262760 25121 sys_catalog.cc:564] T 00000000000000000000000000000000 P 83eae714e78e487d80c3e68e2db3e6d2 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:35:44.263818 25126 sys_catalog.cc:455] T 00000000000000000000000000000000 P 83eae714e78e487d80c3e68e2db3e6d2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "83eae714e78e487d80c3e68e2db3e6d2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83eae714e78e487d80c3e68e2db3e6d2" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 41689 } } }
I20260807 18:35:44.263942 25126 sys_catalog.cc:458] T 00000000000000000000000000000000 P 83eae714e78e487d80c3e68e2db3e6d2 [sys.catalog]: This master's current role is: LEADER
I20260807 18:35:44.264722 25127 sys_catalog.cc:455] T 00000000000000000000000000000000 P 83eae714e78e487d80c3e68e2db3e6d2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 83eae714e78e487d80c3e68e2db3e6d2. Latest consensus state: current_term: 2 leader_uuid: "83eae714e78e487d80c3e68e2db3e6d2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83eae714e78e487d80c3e68e2db3e6d2" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 41689 } } }
I20260807 18:35:44.264837 25127 sys_catalog.cc:458] T 00000000000000000000000000000000 P 83eae714e78e487d80c3e68e2db3e6d2 [sys.catalog]: This master's current role is: LEADER
I20260807 18:35:44.265188 25121 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260807 18:35:44.265228 25134 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:35:44.272500 25134 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:35:44.273268 25134 catalog_manager.cc:1259] Loaded cluster ID: 5c3a69c331b94744bd09d5d5e60d7540
I20260807 18:35:44.273417 25134 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:35:44.276167 25134 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:35:44.276968 25134 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 83eae714e78e487d80c3e68e2db3e6d2: Loaded TSK: 0
I20260807 18:35:44.278334 25134 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 18:35:44.376492 25123 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:35:44.376953 25123 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:35:44.377091 25123 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:35:44.382827 25123 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:35:44.382932 25123 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:35:44.383057 25123 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.14.65
I20260807 18:35:44.388799 25123 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.14.84:36705
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.11.14.84:50947/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.65.keytab
--principal=kudu/127.11.14.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.14.65:37667
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.14.65
--webserver_port=33697
--tserver_master_addrs=127.11.14.126:41689
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25123
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.14.65
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:35:44.390004 25123 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:35:44.391217 25123 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:35:44.402364 25163 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:35:44.402316 25161 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:35:44.402679 25160 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:35:44.402973 25123 server_base.cc:1034] running on GCE node
I20260807 18:35:44.403390 25123 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:35:44.404228 25123 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:35:44.405479 25123 hybrid_clock.cc:648] HybridClock initialized: now 1786127744405449 us; error 40 us; skew 500 ppm
Aug 07 18:35:44 dist-test-slave-6k22 krb5kdc[23989](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127744, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:35:44.409962 25123 init.cc:377] Logged in from keytab as kudu/127.11.14.65@KRBTEST.COM (short username kudu)
I20260807 18:35:44.411149 25123 webserver.cc:460] Webserver started at http://127.11.14.65:33697/ using document root <none> and password file <none>
I20260807 18:35:44.411983 25123 fs_manager.cc:359] Metadata directory not provided
I20260807 18:35:44.412155 25123 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 07 18:35:44 dist-test-slave-6k22 krb5kdc[23989](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127744, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.65@KRBTEST.COM for HTTP/127.11.14.84@KRBTEST.COM
I20260807 18:35:44.445672 25123 fs_manager.cc:709] Time spent opening directory manager: real 0.012s user 0.003s sys 0.000s
I20260807 18:35:44.449681 25171 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:35:44.451470 25123 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 18:35:44.451656 25123 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/minicluster-data/ts-0/data,/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/minicluster-data/ts-0/wal
uuid: "035b879bf04848f3a6c460fb1cb72d52"
format_stamp: "Formatted at 2026-08-07 18:35:42 on dist-test-slave-6k22"
server_key: "708119013fc52ad2e854204f348193fc"
server_key_iv: "927a417be604ce85b4365414960b2b22"
server_key_version: "kuduclusterkey@0"
I20260807 18:35:44.452442 25123 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:35:44.471035 25123 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:35:44.475018 25123 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:35:44.475323 25123 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:35:44.476227 25123 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:35:44.477388 25123 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:35:44.477514 25123 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:35:44.477659 25123 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:35:44.477751 25123 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:35:44.489784 25123 rpc_server.cc:307] RPC server started. Bound to: 127.11.14.65:37667
I20260807 18:35:44.491254 25123 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/minicluster-data/ts-0/data/info.pb
I20260807 18:35:44.494496 25301 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.14.65:37667 every 8 connection(s)
I20260807 18:35:44.498407 11321 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkz56XN/build/debug/bin/kudu as pid 25123
I20260807 18:35:44.498899 11321 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkz56XN/build/debug/bin/kudu
/tmp/dist-test-taskkz56XN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.14.84:50947/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.14.66:39151
--local_ip_for_outbound_sockets=127.11.14.66
--tserver_master_addrs=127.11.14.126:41689
--webserver_port=46525
--webserver_interface=127.11.14.66
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.66.keytab
--principal=kudu/127.11.14.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.14.84:36705
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.11.14.84:50947/kms with env {KRB5CCNAME=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 18:35:44 dist-test-slave-6k22 krb5kdc[23989](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127744, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.65@KRBTEST.COM for kudu/127.11.14.126@KRBTEST.COM
I20260807 18:35:44.535061 25304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:35:44.495385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.65:40923 (local address 127.11.14.126:41689)
0807 18:35:44.496062 (+ 677us) server_negotiation.cc:206] Beginning negotiation
0807 18:35:44.496085 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0807 18:35:44.508276 (+ 12191us) server_negotiation.cc:407] Connection header received
0807 18:35:44.510099 (+ 1823us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:35:44.510120 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:35:44.510625 (+ 505us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:35:44.511039 (+ 414us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:35:44.513220 (+ 2181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:35:44.514856 (+ 1636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:35:44.516705 (+ 1849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:35:44.517486 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:35:44.517639 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:35:44.527961 (+ 10322us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:35:44.527970 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:35:44.527995 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:35:44.528043 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:35:44.530487 (+ 2444us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:35:44.531243 (+ 756us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:35:44.531252 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:35:44.531258 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:35:44.531427 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:35:44.531875 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:35:44.531878 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:35:44.531880 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:35:44.532477 (+ 597us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:35:44.532618 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0807 18:35:44.533493 (+ 875us) server_negotiation.cc:299] Negotiation successful
0807 18:35:44.533860 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":495,"thread_start_us":118,"threads_started":1}
I20260807 18:35:44.537365 25305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:35:44.495540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:41689 (local address 127.11.14.65:40923)
0807 18:35:44.507996 (+ 12456us) negotiation.cc:107] Waiting for socket to connect
0807 18:35:44.508086 (+ 90us) client_negotiation.cc:174] Beginning negotiation
0807 18:35:44.509447 (+ 1361us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:35:44.511796 (+ 2349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:35:44.511808 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:35:44.512254 (+ 446us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:35:44.512990 (+ 736us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:35:44.513015 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:35:44.515056 (+ 2041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:35:44.515059 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:35:44.516536 (+ 1477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:35:44.516549 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:35:44.517946 (+ 1397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:35:44.517950 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:35:44.518154 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:35:44.518975 (+ 821us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:35:44.518999 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:35:44.527599 (+ 8600us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:35:44.530661 (+ 3062us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:35:44.530668 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:35:44.530688 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:35:44.531087 (+ 399us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:35:44.531547 (+ 460us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:35:44.531550 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:35:44.531575 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:35:44.531762 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:35:44.532697 (+ 935us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:35:44.532705 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:35:44.533061 (+ 356us) client_negotiation.cc:769] Sending connection context
0807 18:35:44.536005 (+ 2944us) client_negotiation.cc:240] Negotiation successful
0807 18:35:44.536390 (+ 385us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12112,"thread_start_us":173,"threads_started":1}
I20260807 18:35:44.541062 25302 heartbeater.cc:347] Connected to a master server at 127.11.14.126:41689
I20260807 18:35:44.541347 25302 heartbeater.cc:464] Registering TS with master...
I20260807 18:35:44.541999 25302 heartbeater.cc:511] Master 127.11.14.126:41689 requested a full tablet report, sending...
W20260807 18:35:44.544567 25302 heartbeater.cc:644] Failed to heartbeat to 127.11.14.126:41689 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260807 18:35:44.544698 25302 heartbeater.cc:464] Registering TS with master...
I20260807 18:35:44.544888 25302 heartbeater.cc:511] Master 127.11.14.126:41689 requested a full tablet report, sending...
I20260807 18:35:44.545884 25302 heartbeater.cc:464] Registering TS with master...
I20260807 18:35:44.546064 25302 heartbeater.cc:511] Master 127.11.14.126:41689 requested a full tablet report, sending...
W20260807 18:35:44.546953 25302 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 18:35:44.699080 25306 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:35:44.699530 25306 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:35:44.699669 25306 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:35:44.705428 25306 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:35:44.705576 25306 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:35:44.705705 25306 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.14.66
I20260807 18:35:44.713263 25306 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.14.84:36705
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.11.14.84:50947/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.66.keytab
--principal=kudu/127.11.14.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.14.66:39151
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.14.66
--webserver_port=46525
--tserver_master_addrs=127.11.14.126:41689
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25306
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.14.66
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:35:44.714963 25306 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:35:44.716403 25306 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 18:35:44.730033 25306 server_base.cc:1034] running on GCE node
W20260807 18:35:44.730643 25311 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:35:44.731695 25312 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:35:44.731765 25314 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:35:44.732446 25306 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:35:44.733259 25306 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:35:44.734450 25306 hybrid_clock.cc:648] HybridClock initialized: now 1786127744734417 us; error 50 us; skew 500 ppm
Aug 07 18:35:44 dist-test-slave-6k22 krb5kdc[23989](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127744, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:35:44.738240 25306 init.cc:377] Logged in from keytab as kudu/127.11.14.66@KRBTEST.COM (short username kudu)
I20260807 18:35:44.739437 25306 webserver.cc:460] Webserver started at http://127.11.14.66:46525/ using document root <none> and password file <none>
I20260807 18:35:44.740320 25306 fs_manager.cc:359] Metadata directory not provided
I20260807 18:35:44.740477 25306 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 07 18:35:44 dist-test-slave-6k22 krb5kdc[23989](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127744, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.66@KRBTEST.COM for HTTP/127.11.14.84@KRBTEST.COM
I20260807 18:35:44.759670 25306 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 18:35:44.764431 25322 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:35:44.766125 25306 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.000s
I20260807 18:35:44.766270 25306 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/minicluster-data/ts-1/data,/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/minicluster-data/ts-1/wal
uuid: "1a1fb08cffbd4f619e7354cd653b0157"
format_stamp: "Formatted at 2026-08-07 18:35:43 on dist-test-slave-6k22"
server_key: "d0afd73858240786b5322de26bfae902"
server_key_iv: "e3267af7d87fc8afbb285244cbbb20a7"
server_key_version: "kuduclusterkey@0"
I20260807 18:35:44.766981 25306 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:35:44.789785 25306 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:35:44.793723 25306 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:35:44.793998 25306 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:35:44.794736 25306 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:35:44.795811 25306 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:35:44.795899 25306 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:35:44.795966 25306 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:35:44.796001 25306 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:35:44.806785 25306 rpc_server.cc:307] RPC server started. Bound to: 127.11.14.66:39151
I20260807 18:35:44.808286 25306 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/minicluster-data/ts-1/data/info.pb
I20260807 18:35:44.813648 11321 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkz56XN/build/debug/bin/kudu as pid 25306
I20260807 18:35:44.814154 11321 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkz56XN/build/debug/bin/kudu
/tmp/dist-test-taskkz56XN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.14.84:50947/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.14.67:33565
--local_ip_for_outbound_sockets=127.11.14.67
--tserver_master_addrs=127.11.14.126:41689
--webserver_port=43441
--webserver_interface=127.11.14.67
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.67.keytab
--principal=kudu/127.11.14.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.14.84:36705
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.11.14.84:50947/kms with env {KRB5CCNAME=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 18:35:44.817036 25447 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.14.66:39151 every 8 connection(s)
Aug 07 18:35:44 dist-test-slave-6k22 krb5kdc[23989](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127744, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.66@KRBTEST.COM for kudu/127.11.14.126@KRBTEST.COM
I20260807 18:35:44.874475 25304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:35:44.828154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.66:53813 (local address 127.11.14.126:41689)
0807 18:35:44.828304 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0807 18:35:44.828310 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:35:44.836959 (+ 8649us) server_negotiation.cc:407] Connection header received
0807 18:35:44.852143 (+ 15184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:35:44.852148 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:35:44.852242 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:35:44.852315 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:35:44.859685 (+ 7370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:35:44.860567 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:35:44.863777 (+ 3210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:35:44.864220 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:35:44.864324 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:35:44.868806 (+ 4482us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:35:44.868811 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:35:44.868813 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:35:44.868844 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:35:44.870596 (+ 1752us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:35:44.872449 (+ 1853us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:35:44.872453 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:35:44.872455 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:35:44.872545 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:35:44.873052 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:35:44.873055 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:35:44.873057 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:35:44.873309 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:35:44.873402 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0807 18:35:44.874145 (+ 743us) server_negotiation.cc:299] Negotiation successful
0807 18:35:44.874295 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260807 18:35:44.876698 25453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:35:44.826819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:41689 (local address 127.11.14.66:53813)
0807 18:35:44.836696 (+ 9877us) negotiation.cc:107] Waiting for socket to connect
0807 18:35:44.836806 (+ 110us) client_negotiation.cc:174] Beginning negotiation
0807 18:35:44.849851 (+ 13045us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:35:44.852866 (+ 3015us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:35:44.852879 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:35:44.853282 (+ 403us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:35:44.854063 (+ 781us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:35:44.854091 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:35:44.860809 (+ 6718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:35:44.860814 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:35:44.862311 (+ 1497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:35:44.862327 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:35:44.864660 (+ 2333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:35:44.864664 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:35:44.864896 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:35:44.865585 (+ 689us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:35:44.865607 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:35:44.868568 (+ 2961us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:35:44.871850 (+ 3282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:35:44.871856 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:35:44.871873 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:35:44.872322 (+ 449us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:35:44.872645 (+ 323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:35:44.872648 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:35:44.872650 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:35:44.872928 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:35:44.873483 (+ 555us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:35:44.873489 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:35:44.873815 (+ 326us) client_negotiation.cc:769] Sending connection context
0807 18:35:44.875024 (+ 1209us) client_negotiation.cc:240] Negotiation successful
0807 18:35:44.875384 (+ 360us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8860,"thread_start_us":184,"threads_started":1}
I20260807 18:35:44.878438 25448 heartbeater.cc:347] Connected to a master server at 127.11.14.126:41689
I20260807 18:35:44.878726 25448 heartbeater.cc:464] Registering TS with master...
I20260807 18:35:44.879297 25448 heartbeater.cc:511] Master 127.11.14.126:41689 requested a full tablet report, sending...
W20260807 18:35:44.881186 25448 heartbeater.cc:644] Failed to heartbeat to 127.11.14.126:41689 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260807 18:35:44.881319 25448 heartbeater.cc:464] Registering TS with master...
I20260807 18:35:44.881482 25448 heartbeater.cc:511] Master 127.11.14.126:41689 requested a full tablet report, sending...
I20260807 18:35:44.882402 25448 heartbeater.cc:464] Registering TS with master...
I20260807 18:35:44.882557 25448 heartbeater.cc:511] Master 127.11.14.126:41689 requested a full tablet report, sending...
W20260807 18:35:44.883311 25448 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 18:35:44.990109 25452 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:35:44.990572 25452 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:35:44.990686 25452 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:35:44.996955 25452 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:35:44.997036 25452 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:35:44.997124 25452 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.14.67
I20260807 18:35:45.005057 25452 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.14.84:36705
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.11.14.84:50947/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.67.keytab
--principal=kudu/127.11.14.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.14.67:33565
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.14.67
--webserver_port=43441
--tserver_master_addrs=127.11.14.126:41689
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25452
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.14.67
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:35:45.006515 25452 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:35:45.007925 25452 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:35:45.022969 25461 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:35:45.024550 25458 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:35:45.024576 25459 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:35:45.025650 25452 server_base.cc:1034] running on GCE node
I20260807 18:35:45.026141 25452 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:35:45.026865 25452 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:35:45.028049 25452 hybrid_clock.cc:648] HybridClock initialized: now 1786127745028016 us; error 52 us; skew 500 ppm
Aug 07 18:35:45 dist-test-slave-6k22 krb5kdc[23989](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127745, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:35:45.031426 25452 init.cc:377] Logged in from keytab as kudu/127.11.14.67@KRBTEST.COM (short username kudu)
I20260807 18:35:45.032629 25452 webserver.cc:460] Webserver started at http://127.11.14.67:43441/ using document root <none> and password file <none>
I20260807 18:35:45.033223 25452 fs_manager.cc:359] Metadata directory not provided
I20260807 18:35:45.033331 25452 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 07 18:35:45 dist-test-slave-6k22 krb5kdc[23989](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127745, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.67@KRBTEST.COM for HTTP/127.11.14.84@KRBTEST.COM
I20260807 18:35:45.053987 25452 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.000s sys 0.003s
I20260807 18:35:45.060511 25470 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:35:45.062721 25452 fs_manager.cc:725] Time spent opening block manager: real 0.007s user 0.002s sys 0.001s
I20260807 18:35:45.062884 25452 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/minicluster-data/ts-2/data,/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/minicluster-data/ts-2/wal
uuid: "82c9bc9e8d564f2f963df6aca6e26281"
format_stamp: "Formatted at 2026-08-07 18:35:43 on dist-test-slave-6k22"
server_key: "49cd4511ad56e814e9130b8df790ea6f"
server_key_iv: "1e73e6b6d4d3ba9cf1307ca857919997"
server_key_version: "kuduclusterkey@0"
I20260807 18:35:45.063688 25452 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:35:45.080295 25452 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:35:45.084129 25452 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:35:45.084359 25452 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:35:45.085101 25452 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:35:45.086146 25452 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:35:45.086213 25452 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:35:45.086273 25452 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:35:45.086350 25452 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260807 18:35:45.098251 25452 rpc_server.cc:307] RPC server started. Bound to: 127.11.14.67:33565
I20260807 18:35:45.098312 25597 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.14.67:33565 every 8 connection(s)
I20260807 18:35:45.099399 25452 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/minicluster-data/ts-2/data/info.pb
I20260807 18:35:45.105219 11321 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkz56XN/build/debug/bin/kudu as pid 25452
Aug 07 18:35:45 dist-test-slave-6k22 krb5kdc[23989](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127745, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.67@KRBTEST.COM for kudu/127.11.14.126@KRBTEST.COM
I20260807 18:35:45.116139 25304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:35:45.101645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.67:44079 (local address 127.11.14.126:41689)
0807 18:35:45.101823 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0807 18:35:45.101828 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:35:45.102794 (+ 966us) server_negotiation.cc:407] Connection header received
0807 18:35:45.103831 (+ 1037us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:35:45.103844 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:35:45.103906 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:35:45.104009 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:35:45.105329 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:35:45.106109 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:35:45.107941 (+ 1832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:35:45.108288 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:35:45.108410 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:35:45.111523 (+ 3113us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:35:45.111527 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:35:45.111530 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:35:45.111585 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:35:45.113467 (+ 1882us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:35:45.114150 (+ 683us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:35:45.114154 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:35:45.114156 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:35:45.114259 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:35:45.114679 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:35:45.114682 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:35:45.114684 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:35:45.114914 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:35:45.115039 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0807 18:35:45.115801 (+ 762us) server_negotiation.cc:299] Negotiation successful
0807 18:35:45.115958 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260807 18:35:45.118211 25601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:35:45.102097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:41689 (local address 127.11.14.67:44079)
0807 18:35:45.102591 (+ 494us) negotiation.cc:107] Waiting for socket to connect
0807 18:35:45.102648 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0807 18:35:45.103531 (+ 883us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:35:45.104192 (+ 661us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:35:45.104199 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:35:45.104478 (+ 279us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:35:45.105141 (+ 663us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:35:45.105162 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:35:45.106298 (+ 1136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:35:45.106303 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:35:45.107802 (+ 1499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:35:45.107814 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:35:45.108472 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:35:45.108477 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:35:45.108684 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:35:45.109378 (+ 694us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:35:45.109422 (+ 44us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:35:45.111318 (+ 1896us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:35:45.113617 (+ 2299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:35:45.113623 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:35:45.113642 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:35:45.114023 (+ 381us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:35:45.114383 (+ 360us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:35:45.114386 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:35:45.114388 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:35:45.114578 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:35:45.115065 (+ 487us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:35:45.115072 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:35:45.115427 (+ 355us) client_negotiation.cc:769] Sending connection context
0807 18:35:45.116668 (+ 1241us) client_negotiation.cc:240] Negotiation successful
0807 18:35:45.117062 (+ 394us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":283,"thread_start_us":156,"threads_started":1}
I20260807 18:35:45.119705 25598 heartbeater.cc:347] Connected to a master server at 127.11.14.126:41689
I20260807 18:35:45.119998 25598 heartbeater.cc:464] Registering TS with master...
I20260807 18:35:45.120621 25598 heartbeater.cc:511] Master 127.11.14.126:41689 requested a full tablet report, sending...
W20260807 18:35:45.122771 25598 heartbeater.cc:644] Failed to heartbeat to 127.11.14.126:41689 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260807 18:35:45.122912 25598 heartbeater.cc:464] Registering TS with master...
I20260807 18:35:45.123104 25598 heartbeater.cc:511] Master 127.11.14.126:41689 requested a full tablet report, sending...
I20260807 18:35:45.124213 25082 ts_manager.cc:194] Registered new tserver with Master: 82c9bc9e8d564f2f963df6aca6e26281 (127.11.14.67:33565)
I20260807 18:35:45.125865 25082 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.14.67@KRBTEST.COM'} at 127.11.14.67:44079
I20260807 18:35:45.129782 25602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:35:45.106263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39170 (local address 127.11.14.126:41689)
0807 18:35:45.106548 (+ 285us) server_negotiation.cc:206] Beginning negotiation
0807 18:35:45.106552 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:35:45.106791 (+ 239us) server_negotiation.cc:407] Connection header received
0807 18:35:45.106916 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:35:45.106920 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:35:45.106971 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:35:45.107078 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:35:45.108529 (+ 1451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:35:45.109321 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:35:45.110494 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:35:45.110805 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:35:45.110905 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:35:45.118666 (+ 7761us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:35:45.118669 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:35:45.118671 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:35:45.118719 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:35:45.120860 (+ 2141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:35:45.123712 (+ 2852us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:35:45.123716 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:35:45.123718 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:35:45.123827 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:35:45.124197 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:35:45.124200 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:35:45.124202 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:35:45.129043 (+ 4841us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:35:45.129225 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0807 18:35:45.129467 (+ 242us) server_negotiation.cc:299] Negotiation successful
0807 18:35:45.129609 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":214,"thread_start_us":103,"threads_started":1}
Aug 07 18:35:45 dist-test-slave-6k22 krb5kdc[23989](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786127745, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:35:45.547236 25302 heartbeater.cc:464] Registering TS with master...
I20260807 18:35:45.547621 25302 heartbeater.cc:511] Master 127.11.14.126:41689 requested a full tablet report, sending...
I20260807 18:35:45.548493 25082 ts_manager.cc:194] Registered new tserver with Master: 035b879bf04848f3a6c460fb1cb72d52 (127.11.14.65:37667)
I20260807 18:35:45.549397 25082 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.14.65@KRBTEST.COM'} at 127.11.14.65:40923
I20260807 18:35:45.883579 25448 heartbeater.cc:464] Registering TS with master...
I20260807 18:35:45.883903 25448 heartbeater.cc:511] Master 127.11.14.126:41689 requested a full tablet report, sending...
I20260807 18:35:45.885962 25082 ts_manager.cc:194] Registered new tserver with Master: 1a1fb08cffbd4f619e7354cd653b0157 (127.11.14.66:39151)
I20260807 18:35:45.886725 25082 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.14.66@KRBTEST.COM'} at 127.11.14.66:53813
I20260807 18:35:45.888903 11321 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 18:35:45.900535 25621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:35:45.890658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39178 (local address 127.11.14.126:41689)
0807 18:35:45.891252 (+ 594us) server_negotiation.cc:206] Beginning negotiation
0807 18:35:45.891257 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:35:45.891667 (+ 410us) server_negotiation.cc:407] Connection header received
0807 18:35:45.891714 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:35:45.891717 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:35:45.891766 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:35:45.891875 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:35:45.892934 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:35:45.893760 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:35:45.895002 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:35:45.895372 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:35:45.895506 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:35:45.897034 (+ 1528us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:35:45.897038 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:35:45.897041 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:35:45.897066 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:35:45.898675 (+ 1609us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:35:45.899191 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:35:45.899195 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:35:45.899197 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:35:45.899292 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:35:45.899607 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:35:45.899611 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:35:45.899613 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:35:45.899799 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:35:45.899945 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0807 18:35:45.900190 (+ 245us) server_negotiation.cc:299] Negotiation successful
0807 18:35:45.900314 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":514,"thread_start_us":83,"threads_started":1}
I20260807 18:35:45.904263 25082 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39178:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
Aug 07 18:35:45 dist-test-slave-6k22 krb5kdc[23989](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127745, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.126@KRBTEST.COM for HTTP/127.11.14.84@KRBTEST.COM
W20260807 18:35:45.910982 25082 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 18:35:45.948585 25627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:35:45.930898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.65:37667 (local address 127.0.0.1:50010)
0807 18:35:45.935396 (+ 4498us) negotiation.cc:107] Waiting for socket to connect
0807 18:35:45.935455 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0807 18:35:45.935637 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:35:45.938633 (+ 2996us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:35:45.938640 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:35:45.938666 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:35:45.939071 (+ 405us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:35:45.939079 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:35:45.944949 (+ 5870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:35:45.944952 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:35:45.946018 (+ 1066us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:35:45.946031 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:35:45.947968 (+ 1937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:35:45.947972 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:35:45.948102 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:35:45.948156 (+ 54us) client_negotiation.cc:769] Sending connection context
0807 18:35:45.948254 (+ 98us) client_negotiation.cc:240] Negotiation successful
0807 18:35:45.948410 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4170,"thread_start_us":249,"threads_started":1}
I20260807 18:35:45.950069 25628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:35:45.931039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50010 (local address 127.11.14.65:37667)
0807 18:35:45.936006 (+ 4967us) server_negotiation.cc:206] Beginning negotiation
0807 18:35:45.936016 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 18:35:45.936043 (+ 27us) server_negotiation.cc:407] Connection header received
0807 18:35:45.936128 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:35:45.936134 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:35:45.937958 (+ 1824us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:35:45.938126 (+ 168us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:35:45.939897 (+ 1771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:35:45.944425 (+ 4528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:35:45.946510 (+ 2085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:35:45.947480 (+ 970us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:35:45.949494 (+ 2014us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:35:45.949694 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0807 18:35:45.949755 (+ 61us) server_negotiation.cc:299] Negotiation successful
0807 18:35:45.949856 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4827,"thread_start_us":134,"threads_started":1}
I20260807 18:35:45.950917 25632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:35:45.935093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.66:39151 (local address 127.0.0.1:40866)
0807 18:35:45.937034 (+ 1941us) negotiation.cc:107] Waiting for socket to connect
0807 18:35:45.937058 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0807 18:35:45.937162 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:35:45.938758 (+ 1596us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:35:45.938761 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:35:45.938779 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:35:45.938911 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:35:45.938919 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:35:45.943145 (+ 4226us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:35:45.943149 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:35:45.944773 (+ 1624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:35:45.944786 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:35:45.950290 (+ 5504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:35:45.950293 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:35:45.950386 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:35:45.950411 (+ 25us) client_negotiation.cc:769] Sending connection context
0807 18:35:45.950500 (+ 89us) client_negotiation.cc:240] Negotiation successful
0807 18:35:45.950588 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1881,"thread_start_us":146,"threads_started":1}
I20260807 18:35:45.952921 25633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:35:45.935619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40866 (local address 127.11.14.66:39151)
0807 18:35:45.937411 (+ 1792us) server_negotiation.cc:206] Beginning negotiation
0807 18:35:45.937419 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:35:45.937443 (+ 24us) server_negotiation.cc:407] Connection header received
0807 18:35:45.937527 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:35:45.937533 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:35:45.938198 (+ 665us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:35:45.938379 (+ 181us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:35:45.939229 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:35:45.942977 (+ 3748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:35:45.946860 (+ 3883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:35:45.949903 (+ 3043us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:35:45.950006 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:35:45.950221 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0807 18:35:45.951350 (+ 1129us) server_negotiation.cc:299] Negotiation successful
0807 18:35:45.951463 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1675,"thread_start_us":85,"threads_started":1}
I20260807 18:35:45.954193 25222 tablet_service.cc:1467] Processing CreateTablet for tablet 7248b7e56515433081df4abd3ec301e3 (DEFAULT_TABLE table=test-table [id=2be76a81ab7f4d32acb8a5fb03964238]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:35:45.955523 25222 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7248b7e56515433081df4abd3ec301e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:35:45.956203 25375 tablet_service.cc:1467] Processing CreateTablet for tablet 7248b7e56515433081df4abd3ec301e3 (DEFAULT_TABLE table=test-table [id=2be76a81ab7f4d32acb8a5fb03964238]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:35:45.957327 25375 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7248b7e56515433081df4abd3ec301e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:35:45.957648 25629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:35:45.933148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.67:33565 (local address 127.0.0.1:54390)
0807 18:35:45.935659 (+ 2511us) negotiation.cc:107] Waiting for socket to connect
0807 18:35:45.935674 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 18:35:45.935799 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:35:45.947909 (+ 12110us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:35:45.947913 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:35:45.947936 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:35:45.948038 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:35:45.948044 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:35:45.954773 (+ 6729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:35:45.954776 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:35:45.956003 (+ 1227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:35:45.956016 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:35:45.957237 (+ 1221us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:35:45.957241 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:35:45.957357 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:35:45.957388 (+ 31us) client_negotiation.cc:769] Sending connection context
0807 18:35:45.957444 (+ 56us) client_negotiation.cc:240] Negotiation successful
0807 18:35:45.957516 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2437,"thread_start_us":231,"threads_started":1}
I20260807 18:35:45.958508 25631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:35:45.934165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54390 (local address 127.11.14.67:33565)
0807 18:35:45.945484 (+ 11319us) server_negotiation.cc:206] Beginning negotiation
0807 18:35:45.945492 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:35:45.945520 (+ 28us) server_negotiation.cc:407] Connection header received
0807 18:35:45.945604 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:35:45.945610 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:35:45.947754 (+ 2144us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:35:45.948126 (+ 372us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:35:45.948287 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:35:45.954630 (+ 6343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:35:45.956215 (+ 1585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:35:45.957079 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:35:45.957919 (+ 840us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:35:45.958162 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0807 18:35:45.958224 (+ 62us) server_negotiation.cc:299] Negotiation successful
0807 18:35:45.958333 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11178,"thread_start_us":175,"threads_started":1}
I20260807 18:35:45.961548 25523 tablet_service.cc:1467] Processing CreateTablet for tablet 7248b7e56515433081df4abd3ec301e3 (DEFAULT_TABLE table=test-table [id=2be76a81ab7f4d32acb8a5fb03964238]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:35:45.962620 25523 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7248b7e56515433081df4abd3ec301e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:35:45.971457 25635 tablet_bootstrap.cc:492] T 7248b7e56515433081df4abd3ec301e3 P 82c9bc9e8d564f2f963df6aca6e26281: Bootstrap starting.
I20260807 18:35:45.973991 25635 tablet_bootstrap.cc:654] T 7248b7e56515433081df4abd3ec301e3 P 82c9bc9e8d564f2f963df6aca6e26281: Neither blocks nor log segments found. Creating new log.
I20260807 18:35:45.975435 25635 log.cc:824] T 7248b7e56515433081df4abd3ec301e3 P 82c9bc9e8d564f2f963df6aca6e26281: Log is configured to *not* fsync() on all Append() calls
I20260807 18:35:45.977970 25635 tablet_bootstrap.cc:492] T 7248b7e56515433081df4abd3ec301e3 P 82c9bc9e8d564f2f963df6aca6e26281: No bootstrap required, opened a new log
I20260807 18:35:45.978209 25635 ts_tablet_manager.cc:1397] T 7248b7e56515433081df4abd3ec301e3 P 82c9bc9e8d564f2f963df6aca6e26281: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260807 18:35:45.982580 25634 tablet_bootstrap.cc:492] T 7248b7e56515433081df4abd3ec301e3 P 1a1fb08cffbd4f619e7354cd653b0157: Bootstrap starting.
I20260807 18:35:45.985896 25635 raft_consensus.cc:348] T 7248b7e56515433081df4abd3ec301e3 P 82c9bc9e8d564f2f963df6aca6e26281 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "035b879bf04848f3a6c460fb1cb72d52" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 37667 } } peers { permanent_uuid: "82c9bc9e8d564f2f963df6aca6e26281" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 33565 } } peers { permanent_uuid: "1a1fb08cffbd4f619e7354cd653b0157" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 39151 } }
I20260807 18:35:45.986265 25635 raft_consensus.cc:374] T 7248b7e56515433081df4abd3ec301e3 P 82c9bc9e8d564f2f963df6aca6e26281 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:35:45.986150 25634 tablet_bootstrap.cc:654] T 7248b7e56515433081df4abd3ec301e3 P 1a1fb08cffbd4f619e7354cd653b0157: Neither blocks nor log segments found. Creating new log.
I20260807 18:35:45.986361 25635 raft_consensus.cc:729] T 7248b7e56515433081df4abd3ec301e3 P 82c9bc9e8d564f2f963df6aca6e26281 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 82c9bc9e8d564f2f963df6aca6e26281, State: Initialized, Role: FOLLOWER
I20260807 18:35:45.986898 25635 consensus_queue.cc:260] T 7248b7e56515433081df4abd3ec301e3 P 82c9bc9e8d564f2f963df6aca6e26281 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "035b879bf04848f3a6c460fb1cb72d52" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 37667 } } peers { permanent_uuid: "82c9bc9e8d564f2f963df6aca6e26281" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 33565 } } peers { permanent_uuid: "1a1fb08cffbd4f619e7354cd653b0157" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 39151 } }
I20260807 18:35:45.987953 25634 log.cc:824] T 7248b7e56515433081df4abd3ec301e3 P 1a1fb08cffbd4f619e7354cd653b0157: Log is configured to *not* fsync() on all Append() calls
I20260807 18:35:45.988127 25635 ts_tablet_manager.cc:1428] T 7248b7e56515433081df4abd3ec301e3 P 82c9bc9e8d564f2f963df6aca6e26281: Time spent starting tablet: real 0.010s user 0.003s sys 0.004s
I20260807 18:35:45.988629 25598 heartbeater.cc:503] Master 127.11.14.126:41689 was elected leader, sending a full tablet report...
I20260807 18:35:45.991016 25634 tablet_bootstrap.cc:492] T 7248b7e56515433081df4abd3ec301e3 P 1a1fb08cffbd4f619e7354cd653b0157: No bootstrap required, opened a new log
I20260807 18:35:45.991228 25634 ts_tablet_manager.cc:1397] T 7248b7e56515433081df4abd3ec301e3 P 1a1fb08cffbd4f619e7354cd653b0157: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260807 18:35:45.997330 25634 raft_consensus.cc:348] T 7248b7e56515433081df4abd3ec301e3 P 1a1fb08cffbd4f619e7354cd653b0157 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "035b879bf04848f3a6c460fb1cb72d52" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 37667 } } peers { permanent_uuid: "82c9bc9e8d564f2f963df6aca6e26281" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 33565 } } peers { permanent_uuid: "1a1fb08cffbd4f619e7354cd653b0157" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 39151 } }
I20260807 18:35:45.997679 25634 raft_consensus.cc:374] T 7248b7e56515433081df4abd3ec301e3 P 1a1fb08cffbd4f619e7354cd653b0157 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:35:45.997849 25634 raft_consensus.cc:729] T 7248b7e56515433081df4abd3ec301e3 P 1a1fb08cffbd4f619e7354cd653b0157 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a1fb08cffbd4f619e7354cd653b0157, State: Initialized, Role: FOLLOWER
I20260807 18:35:45.998423 25634 consensus_queue.cc:260] T 7248b7e56515433081df4abd3ec301e3 P 1a1fb08cffbd4f619e7354cd653b0157 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "035b879bf04848f3a6c460fb1cb72d52" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 37667 } } peers { permanent_uuid: "82c9bc9e8d564f2f963df6aca6e26281" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 33565 } } peers { permanent_uuid: "1a1fb08cffbd4f619e7354cd653b0157" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 39151 } }
I20260807 18:35:45.999336 25448 heartbeater.cc:503] Master 127.11.14.126:41689 was elected leader, sending a full tablet report...
I20260807 18:35:46.002525 25634 ts_tablet_manager.cc:1428] T 7248b7e56515433081df4abd3ec301e3 P 1a1fb08cffbd4f619e7354cd653b0157: Time spent starting tablet: real 0.011s user 0.005s sys 0.004s
I20260807 18:35:46.003222 25637 tablet_bootstrap.cc:492] T 7248b7e56515433081df4abd3ec301e3 P 035b879bf04848f3a6c460fb1cb72d52: Bootstrap starting.
I20260807 18:35:46.006042 25637 tablet_bootstrap.cc:654] T 7248b7e56515433081df4abd3ec301e3 P 035b879bf04848f3a6c460fb1cb72d52: Neither blocks nor log segments found. Creating new log.
I20260807 18:35:46.007031 25637 log.cc:824] T 7248b7e56515433081df4abd3ec301e3 P 035b879bf04848f3a6c460fb1cb72d52: Log is configured to *not* fsync() on all Append() calls
I20260807 18:35:46.009981 25637 tablet_bootstrap.cc:492] T 7248b7e56515433081df4abd3ec301e3 P 035b879bf04848f3a6c460fb1cb72d52: No bootstrap required, opened a new log
I20260807 18:35:46.010207 25637 ts_tablet_manager.cc:1397] T 7248b7e56515433081df4abd3ec301e3 P 035b879bf04848f3a6c460fb1cb72d52: Time spent bootstrapping tablet: real 0.007s user 0.007s sys 0.000s
I20260807 18:35:46.014564 25637 raft_consensus.cc:348] T 7248b7e56515433081df4abd3ec301e3 P 035b879bf04848f3a6c460fb1cb72d52 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "035b879bf04848f3a6c460fb1cb72d52" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 37667 } } peers { permanent_uuid: "82c9bc9e8d564f2f963df6aca6e26281" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 33565 } } peers { permanent_uuid: "1a1fb08cffbd4f619e7354cd653b0157" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 39151 } }
I20260807 18:35:46.014879 25637 raft_consensus.cc:374] T 7248b7e56515433081df4abd3ec301e3 P 035b879bf04848f3a6c460fb1cb72d52 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:35:46.014951 25637 raft_consensus.cc:729] T 7248b7e56515433081df4abd3ec301e3 P 035b879bf04848f3a6c460fb1cb72d52 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 035b879bf04848f3a6c460fb1cb72d52, State: Initialized, Role: FOLLOWER
I20260807 18:35:46.015722 25637 consensus_queue.cc:260] T 7248b7e56515433081df4abd3ec301e3 P 035b879bf04848f3a6c460fb1cb72d52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "035b879bf04848f3a6c460fb1cb72d52" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 37667 } } peers { permanent_uuid: "82c9bc9e8d564f2f963df6aca6e26281" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 33565 } } peers { permanent_uuid: "1a1fb08cffbd4f619e7354cd653b0157" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 39151 } }
I20260807 18:35:46.017022 25637 ts_tablet_manager.cc:1428] T 7248b7e56515433081df4abd3ec301e3 P 035b879bf04848f3a6c460fb1cb72d52: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260807 18:35:46.017464 25302 heartbeater.cc:503] Master 127.11.14.126:41689 was elected leader, sending a full tablet report...
I20260807 18:35:46.053818 25640 raft_consensus.cc:482] T 7248b7e56515433081df4abd3ec301e3 P 1a1fb08cffbd4f619e7354cd653b0157 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:35:46.054129 25640 raft_consensus.cc:504] T 7248b7e56515433081df4abd3ec301e3 P 1a1fb08cffbd4f619e7354cd653b0157 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "035b879bf04848f3a6c460fb1cb72d52" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 37667 } } peers { permanent_uuid: "82c9bc9e8d564f2f963df6aca6e26281" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 33565 } } peers { permanent_uuid: "1a1fb08cffbd4f619e7354cd653b0157" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 39151 } }
I20260807 18:35:46.055639 25640 leader_election.cc:288] T 7248b7e56515433081df4abd3ec301e3 P 1a1fb08cffbd4f619e7354cd653b0157 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 035b879bf04848f3a6c460fb1cb72d52 (127.11.14.65:37667), 82c9bc9e8d564f2f963df6aca6e26281 (127.11.14.67:33565)
I20260807 18:35:46.061393 25631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:35:46.056435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.66:56275 (local address 127.11.14.67:33565)
0807 18:35:46.056569 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0807 18:35:46.056573 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:35:46.057078 (+ 505us) server_negotiation.cc:407] Connection header received
0807 18:35:46.057291 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:35:46.057295 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:35:46.057343 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:35:46.057391 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:35:46.057814 (+ 423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:35:46.058519 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:35:46.060059 (+ 1540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:35:46.060634 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:35:46.060733 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:35:46.060791 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0807 18:35:46.061153 (+ 362us) server_negotiation.cc:299] Negotiation successful
0807 18:35:46.061248 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260807 18:35:46.061549 25628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:35:46.056372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.66:47285 (local address 127.11.14.65:37667)
0807 18:35:46.056497 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0807 18:35:46.056502 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:35:46.056807 (+ 305us) server_negotiation.cc:407] Connection header received
0807 18:35:46.056967 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:35:46.056970 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:35:46.057020 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:35:46.057066 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:35:46.057491 (+ 425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:35:46.058334 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:35:46.060222 (+ 1888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:35:46.060823 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:35:46.060929 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:35:46.060998 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0807 18:35:46.061347 (+ 349us) server_negotiation.cc:299] Negotiation successful
0807 18:35:46.061425 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260807 18:35:46.062044 25644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:35:46.055856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.65:37667 (local address 127.11.14.66:47285)
0807 18:35:46.056685 (+ 829us) negotiation.cc:107] Waiting for socket to connect
0807 18:35:46.056701 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 18:35:46.056815 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:35:46.057216 (+ 401us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:35:46.057219 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:35:46.057235 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:35:46.057374 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:35:46.057380 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:35:46.058561 (+ 1181us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:35:46.058564 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:35:46.060070 (+ 1506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:35:46.060081 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:35:46.061095 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:35:46.061099 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:35:46.061212 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:35:46.061233 (+ 21us) client_negotiation.cc:769] Sending connection context
0807 18:35:46.061762 (+ 529us) client_negotiation.cc:240] Negotiation successful
0807 18:35:46.061842 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":760,"mutex_wait_us":80,"thread_start_us":151,"threads_started":1}
I20260807 18:35:46.063059 25246 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7248b7e56515433081df4abd3ec301e3" candidate_uuid: "1a1fb08cffbd4f619e7354cd653b0157" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "035b879bf04848f3a6c460fb1cb72d52" is_pre_election: true
I20260807 18:35:46.063464 25246 raft_consensus.cc:2454] T 7248b7e56515433081df4abd3ec301e3 P 035b879bf04848f3a6c460fb1cb72d52 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1a1fb08cffbd4f619e7354cd653b0157 in term 0.
I20260807 18:35:46.062204 25643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:35:46.055997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.67:33565 (local address 127.11.14.66:56275)
0807 18:35:46.056987 (+ 990us) negotiation.cc:107] Waiting for socket to connect
0807 18:35:46.057002 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 18:35:46.057153 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:35:46.057531 (+ 378us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:35:46.057534 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:35:46.057550 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:35:46.057676 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:35:46.057682 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:35:46.058666 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:35:46.058669 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:35:46.059909 (+ 1240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:35:46.059920 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:35:46.060897 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:35:46.060900 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:35:46.060997 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:35:46.061022 (+ 25us) client_negotiation.cc:769] Sending connection context
0807 18:35:46.061934 (+ 912us) client_negotiation.cc:240] Negotiation successful
0807 18:35:46.062017 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":923,"thread_start_us":123,"threads_started":1}
I20260807 18:35:46.064292 25326 leader_election.cc:302] T 7248b7e56515433081df4abd3ec301e3 P 1a1fb08cffbd4f619e7354cd653b0157 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 035b879bf04848f3a6c460fb1cb72d52, 1a1fb08cffbd4f619e7354cd653b0157; no voters:
I20260807 18:35:46.064400 25526 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7248b7e56515433081df4abd3ec301e3" candidate_uuid: "1a1fb08cffbd4f619e7354cd653b0157" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "82c9bc9e8d564f2f963df6aca6e26281" is_pre_election: true
I20260807 18:35:46.064643 25640 raft_consensus.cc:2781] T 7248b7e56515433081df4abd3ec301e3 P 1a1fb08cffbd4f619e7354cd653b0157 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:35:46.064677 25526 raft_consensus.cc:2454] T 7248b7e56515433081df4abd3ec301e3 P 82c9bc9e8d564f2f963df6aca6e26281 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1a1fb08cffbd4f619e7354cd653b0157 in term 0.
I20260807 18:35:46.064725 25640 raft_consensus.cc:482] T 7248b7e56515433081df4abd3ec301e3 P 1a1fb08cffbd4f619e7354cd653b0157 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:35:46.064767 25640 raft_consensus.cc:3037] T 7248b7e56515433081df4abd3ec301e3 P 1a1fb08cffbd4f619e7354cd653b0157 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:35:46.066202 25640 raft_consensus.cc:504] T 7248b7e56515433081df4abd3ec301e3 P 1a1fb08cffbd4f619e7354cd653b0157 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "035b879bf04848f3a6c460fb1cb72d52" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 37667 } } peers { permanent_uuid: "82c9bc9e8d564f2f963df6aca6e26281" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 33565 } } peers { permanent_uuid: "1a1fb08cffbd4f619e7354cd653b0157" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 39151 } }
I20260807 18:35:46.066602 25640 leader_election.cc:288] T 7248b7e56515433081df4abd3ec301e3 P 1a1fb08cffbd4f619e7354cd653b0157 [CANDIDATE]: Term 1 election: Requested vote from peers 035b879bf04848f3a6c460fb1cb72d52 (127.11.14.65:37667), 82c9bc9e8d564f2f963df6aca6e26281 (127.11.14.67:33565)
I20260807 18:35:46.066957 25246 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7248b7e56515433081df4abd3ec301e3" candidate_uuid: "1a1fb08cffbd4f619e7354cd653b0157" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "035b879bf04848f3a6c460fb1cb72d52"
I20260807 18:35:46.067078 25246 raft_consensus.cc:3037] T 7248b7e56515433081df4abd3ec301e3 P 035b879bf04848f3a6c460fb1cb72d52 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:35:46.067732 25526 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7248b7e56515433081df4abd3ec301e3" candidate_uuid: "1a1fb08cffbd4f619e7354cd653b0157" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "82c9bc9e8d564f2f963df6aca6e26281"
I20260807 18:35:46.067860 25526 raft_consensus.cc:3037] T 7248b7e56515433081df4abd3ec301e3 P 82c9bc9e8d564f2f963df6aca6e26281 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:35:46.068434 25246 raft_consensus.cc:2454] T 7248b7e56515433081df4abd3ec301e3 P 035b879bf04848f3a6c460fb1cb72d52 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1a1fb08cffbd4f619e7354cd653b0157 in term 1.
I20260807 18:35:46.068832 25326 leader_election.cc:302] T 7248b7e56515433081df4abd3ec301e3 P 1a1fb08cffbd4f619e7354cd653b0157 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 035b879bf04848f3a6c460fb1cb72d52, 1a1fb08cffbd4f619e7354cd653b0157; no voters:
I20260807 18:35:46.069211 25526 raft_consensus.cc:2454] T 7248b7e56515433081df4abd3ec301e3 P 82c9bc9e8d564f2f963df6aca6e26281 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1a1fb08cffbd4f619e7354cd653b0157 in term 1.
I20260807 18:35:46.069521 25640 raft_consensus.cc:2781] T 7248b7e56515433081df4abd3ec301e3 P 1a1fb08cffbd4f619e7354cd653b0157 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:35:46.069782 25640 raft_consensus.cc:686] T 7248b7e56515433081df4abd3ec301e3 P 1a1fb08cffbd4f619e7354cd653b0157 [term 1 LEADER]: Becoming Leader. State: Replica: 1a1fb08cffbd4f619e7354cd653b0157, State: Running, Role: LEADER
I20260807 18:35:46.070143 25640 consensus_queue.cc:237] T 7248b7e56515433081df4abd3ec301e3 P 1a1fb08cffbd4f619e7354cd653b0157 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "035b879bf04848f3a6c460fb1cb72d52" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 37667 } } peers { permanent_uuid: "82c9bc9e8d564f2f963df6aca6e26281" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 33565 } } peers { permanent_uuid: "1a1fb08cffbd4f619e7354cd653b0157" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 39151 } }
I20260807 18:35:46.073853 25081 catalog_manager.cc:5495] T 7248b7e56515433081df4abd3ec301e3 P 1a1fb08cffbd4f619e7354cd653b0157 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1a1fb08cffbd4f619e7354cd653b0157 (127.11.14.66). New cstate: current_term: 1 leader_uuid: "1a1fb08cffbd4f619e7354cd653b0157" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "035b879bf04848f3a6c460fb1cb72d52" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 37667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "82c9bc9e8d564f2f963df6aca6e26281" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 33565 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1a1fb08cffbd4f619e7354cd653b0157" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 39151 } health_report { overall_health: HEALTHY } } }
W20260807 18:35:46.075098 25449 tablet.cc:2369] T 7248b7e56515433081df4abd3ec301e3 P 1a1fb08cffbd4f619e7354cd653b0157: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:35:46.094041 25633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:35:46.089615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40874 (local address 127.11.14.66:39151)
0807 18:35:46.090027 (+ 412us) server_negotiation.cc:206] Beginning negotiation
0807 18:35:46.090032 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:35:46.090046 (+ 14us) server_negotiation.cc:407] Connection header received
0807 18:35:46.090145 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:35:46.090148 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:35:46.090219 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:35:46.090345 (+ 126us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:35:46.090720 (+ 375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:35:46.091265 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:35:46.092331 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:35:46.092575 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:35:46.092698 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:35:46.093096 (+ 398us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:35:46.093539 (+ 443us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:35:46.093712 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0807 18:35:46.093782 (+ 70us) server_negotiation.cc:299] Negotiation successful
0807 18:35:46.093845 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":310}
W20260807 18:35:46.101184 25599 tablet.cc:2369] T 7248b7e56515433081df4abd3ec301e3 P 82c9bc9e8d564f2f963df6aca6e26281: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:35:46.101620 25526 raft_consensus.cc:1261] T 7248b7e56515433081df4abd3ec301e3 P 82c9bc9e8d564f2f963df6aca6e26281 [term 1 FOLLOWER]: Refusing update from remote peer 1a1fb08cffbd4f619e7354cd653b0157: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:35:46.102926 25246 raft_consensus.cc:1261] T 7248b7e56515433081df4abd3ec301e3 P 035b879bf04848f3a6c460fb1cb72d52 [term 1 FOLLOWER]: Refusing update from remote peer 1a1fb08cffbd4f619e7354cd653b0157: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:35:46.103741 25640 consensus_queue.cc:1035] T 7248b7e56515433081df4abd3ec301e3 P 1a1fb08cffbd4f619e7354cd653b0157 [LEADER]: Connected to new peer: Peer: permanent_uuid: "82c9bc9e8d564f2f963df6aca6e26281" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 33565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:35:46.104180 25640 consensus_queue.cc:1035] T 7248b7e56515433081df4abd3ec301e3 P 1a1fb08cffbd4f619e7354cd653b0157 [LEADER]: Connected to new peer: Peer: permanent_uuid: "035b879bf04848f3a6c460fb1cb72d52" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 37667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:35:46.114666 25648 mvcc.cc:205] Tried to move back new op lower bound from 7315979248020688896 to 7315979247906459648. Current Snapshot: MvccSnapshot[applied={T|T < 7315979248020688896}]
I20260807 18:35:46.135370 25081 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39178:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:35:46.135735 25081 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39178:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:35:46.138950 25081 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 83eae714e78e487d80c3e68e2db3e6d2: Sending DeleteTablet for 3 replicas of tablet 7248b7e56515433081df4abd3ec301e3
I20260807 18:35:46.139937 25222 tablet_service.cc:1514] Processing DeleteTablet for tablet 7248b7e56515433081df4abd3ec301e3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:35:46 UTC) from {username='kudu', principal='kudu/127.11.14.126@KRBTEST.COM'} at 127.0.0.1:50010
I20260807 18:35:46.141088 25523 tablet_service.cc:1514] Processing DeleteTablet for tablet 7248b7e56515433081df4abd3ec301e3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:35:46 UTC) from {username='kudu', principal='kudu/127.11.14.126@KRBTEST.COM'} at 127.0.0.1:54390
I20260807 18:35:46.142143 25375 tablet_service.cc:1514] Processing DeleteTablet for tablet 7248b7e56515433081df4abd3ec301e3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:35:46 UTC) from {username='kudu', principal='kudu/127.11.14.126@KRBTEST.COM'} at 127.0.0.1:40866
I20260807 18:35:46.143639 11321 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkz56XN/build/debug/bin/kudu with pid 25123
I20260807 18:35:46.151302 25659 tablet_replica.cc:330] T 7248b7e56515433081df4abd3ec301e3 P 1a1fb08cffbd4f619e7354cd653b0157: stopping tablet replica
I20260807 18:35:46.151592 25658 tablet_replica.cc:330] T 7248b7e56515433081df4abd3ec301e3 P 82c9bc9e8d564f2f963df6aca6e26281: stopping tablet replica
I20260807 18:35:46.152020 25658 raft_consensus.cc:2229] T 7248b7e56515433081df4abd3ec301e3 P 82c9bc9e8d564f2f963df6aca6e26281 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:35:46.152343 25659 raft_consensus.cc:2229] T 7248b7e56515433081df4abd3ec301e3 P 1a1fb08cffbd4f619e7354cd653b0157 [term 1 LEADER]: Raft consensus shutting down.
I20260807 18:35:46.152776 25659 raft_consensus.cc:2258] T 7248b7e56515433081df4abd3ec301e3 P 1a1fb08cffbd4f619e7354cd653b0157 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:35:46.153154 25658 raft_consensus.cc:2258] T 7248b7e56515433081df4abd3ec301e3 P 82c9bc9e8d564f2f963df6aca6e26281 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:35:46.155371 25659 ts_tablet_manager.cc:1905] T 7248b7e56515433081df4abd3ec301e3 P 1a1fb08cffbd4f619e7354cd653b0157: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:35:46.156508 25658 ts_tablet_manager.cc:1905] T 7248b7e56515433081df4abd3ec301e3 P 82c9bc9e8d564f2f963df6aca6e26281: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:35:46.160382 25658 ts_tablet_manager.cc:1918] T 7248b7e56515433081df4abd3ec301e3 P 82c9bc9e8d564f2f963df6aca6e26281: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
W20260807 18:35:46.161768 25067 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.14.65:37667 (error 108)
W20260807 18:35:46.162976 25629 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:35:46.162612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.65:37667 (local address 127.0.0.1:50018)
0807 18:35:46.162759 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0807 18:35:46.162776 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 18:35:46.162862 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.65:37667: BlockingWrite error: write error: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":57}
W20260807 18:35:46.163146 25067 catalog_manager.cc:4584] TS 035b879bf04848f3a6c460fb1cb72d52 (127.11.14.65:37667): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 7248b7e56515433081df4abd3ec301e3: Network error: Client connection negotiation failed: client connection to 127.11.14.65:37667: BlockingWrite error: write error: Connection reset by peer (error 104)
I20260807 18:35:46.163456 25658 log.cc:1196] T 7248b7e56515433081df4abd3ec301e3 P 82c9bc9e8d564f2f963df6aca6e26281: Deleting WAL directory at /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786127579900523-11321-0/minicluster-data/ts-2/wal/wals/7248b7e56515433081df4abd3ec301e3
I20260807 18:35:46.163543 11321 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkz56XN/build/debug/bin/kudu with pid 25306
I20260807 18:35:46.170004 25658 ts_tablet_manager.cc:1939] T 7248b7e56515433081df4abd3ec301e3 P 82c9bc9e8d564f2f963df6aca6e26281: Deleting consensus metadata
I20260807 18:35:46.171741 25065 catalog_manager.cc:4841] TS 82c9bc9e8d564f2f963df6aca6e26281 (127.11.14.67:33565): tablet 7248b7e56515433081df4abd3ec301e3 (table test-table [id=2be76a81ab7f4d32acb8a5fb03964238]) successfully deleted
W20260807 18:35:46.179116 25629 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:35:46.178448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.66:39151 (local address 127.0.0.1:40886)
0807 18:35:46.178648 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0807 18:35:46.178663 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 18:35:46.178773 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:35:46.179008 (+ 235us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.66:39151: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":111}
I20260807 18:35:46.179736 11321 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkz56XN/build/debug/bin/kudu with pid 25452
I20260807 18:35:46.189445 11321 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkz56XN/build/debug/bin/kudu with pid 25045
I20260807 18:35:46.197108 11321 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260807 18:35:46.542235 11321 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260807 18:35:46.542399 11321 mini_ranger.cc:66] Stopping Ranger...
I20260807 18:35:46.888549 11321 mini_ranger.cc:68] Stopped Ranger
2026-08-07 18:35:46.888 UTC [24022] LOG: received smart shutdown request
2026-08-07 18:35:46.893 UTC [24022] LOG: background worker "logical replication launcher" (PID 24033) exited with exit code 1
2026-08-07 18:35:46.894 UTC [24026] LOG: shutting down
2026-08-07 18:35:47.079 UTC [24022] LOG: database system is shut down
2026-08-07T18:35:47Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (50341 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786127579900523-11321-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:35:47 dist-test-slave-6k22 krb5kdc[25671](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:35:47 dist-test-slave-6k22 krb5kdc[25671](info): set up 2 sockets
Aug 07 18:35:47 dist-test-slave-6k22 krb5kdc[25671](info): commencing operation
krb5kdc: starting...
W20260807 18:35:48.396039 11321 mini_kdc.cc:121] Time spent starting KDC: real 1.172s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786127579900523-11321-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786127579900523-11321-0/krb5kdc/test-user.keytab.
Aug 07 18:35:48 dist-test-slave-6k22 krb5kdc[25671](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127748, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:35:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:35:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.14.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.14.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.14.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.126.keytab.
Entry for principal kudu/127.11.14.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.126.keytab.
I20260807 18:35:48.558110 11321 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkz56XN/build/debug/bin/kudu
/tmp/dist-test-taskkz56XN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786127579900523-11321-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786127579900523-11321-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786127579900523-11321-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786127579900523-11321-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.14.126:40395
--webserver_interface=127.11.14.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.126.keytab
--principal=kudu/127.11.14.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.14.84:34377
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.14.126:40395,127.11.14.125:36793,127.11.14.124:37595
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786127579900523-11321-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786127579900523-11321-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786127579900523-11321-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:35:48.677187 25686 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:35:48.677491 25686 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:35:48.677534 25686 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:35:48.681778 25686 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:35:48.681855 25686 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:35:48.681875 25686 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:35:48.681892 25686 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:35:48.681955 25686 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:35:48.688144 25686 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.14.84:34377
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786127579900523-11321-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786127579900523-11321-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.14.126:40395,127.11.14.125:36793,127.11.14.124:37595
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.126.keytab
--principal=kudu/127.11.14.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.14.126:40395
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786127579900523-11321-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.14.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25686
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786127579900523-11321-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:35:48.689391 25686 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:35:48.690516 25686 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:35:48.699703 25696 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:35:48.699892 25686 server_base.cc:1034] running on GCE node
W20260807 18:35:48.699724 25693 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:35:48.700139 25692 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:35:48.700740 25686 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:35:48.701912 25686 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:35:48.703118 25686 hybrid_clock.cc:648] HybridClock initialized: now 1786127748703096 us; error 33 us; skew 500 ppm
Aug 07 18:35:48 dist-test-slave-6k22 krb5kdc[25671](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127748, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:35:48.706131 25686 init.cc:377] Logged in from keytab as kudu/127.11.14.126@KRBTEST.COM (short username kudu)
I20260807 18:35:48.707072 25686 webserver.cc:460] Webserver started at http://127.11.14.126:36081/ using document root <none> and password file <none>
I20260807 18:35:48.707739 25686 fs_manager.cc:359] Metadata directory not provided
I20260807 18:35:48.707846 25686 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:35:48.708173 25686 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:35:48.710323 25686 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786127579900523-11321-0/minicluster-data/master-0/data/instance:
uuid: "5532e9367168440c949f3e3a203cef06"
format_stamp: "Formatted at 2026-08-07 18:35:48 on dist-test-slave-6k22"
server_key: "59e2ccc308587294a6f4c66b197a2e03"
server_key_iv: "d35cdb6ba55d3c511801790b009b1373"
server_key_version: "encryptionkey@0"
I20260807 18:35:48.710928 25686 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786127579900523-11321-0/minicluster-data/master-0/wal/instance:
uuid: "5532e9367168440c949f3e3a203cef06"
format_stamp: "Formatted at 2026-08-07 18:35:48 on dist-test-slave-6k22"
server_key: "59e2ccc308587294a6f4c66b197a2e03"
server_key_iv: "d35cdb6ba55d3c511801790b009b1373"
server_key_version: "encryptionkey@0"
I20260807 18:35:48.714821 25686 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.003s
I20260807 18:35:48.717423 25705 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:35:48.718706 25686 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260807 18:35:48.718864 25686 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786127579900523-11321-0/minicluster-data/master-0/data,/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786127579900523-11321-0/minicluster-data/master-0/wal
uuid: "5532e9367168440c949f3e3a203cef06"
format_stamp: "Formatted at 2026-08-07 18:35:48 on dist-test-slave-6k22"
server_key: "59e2ccc308587294a6f4c66b197a2e03"
server_key_iv: "d35cdb6ba55d3c511801790b009b1373"
server_key_version: "encryptionkey@0"
I20260807 18:35:48.718986 25686 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786127579900523-11321-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786127579900523-11321-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786127579900523-11321-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:35:48.756852 25686 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:35:48.760133 25686 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:35:48.760367 25686 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:35:48.767865 25686 rpc_server.cc:307] RPC server started. Bound to: 127.11.14.126:40395
I20260807 18:35:48.767886 25762 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.14.126:40395 every 8 connection(s)
I20260807 18:35:48.768882 25686 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786127579900523-11321-0/minicluster-data/master-0/data/info.pb
I20260807 18:35:48.772085 25763 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:35:48.776662 11321 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkz56XN/build/debug/bin/kudu as pid 25686
I20260807 18:35:48.776804 11321 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786127579900523-11321-0/minicluster-data/master-0/wal/instance
I20260807 18:35:48.776449 25763 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.14.126" port: 40395 } has no permanent_uuid. Determining permanent_uuid...
I20260807 18:35:48.777150 11321 external_mini_cluster.cc:1428] Setting key 73c8e6e9227258be8cdeec4133500429
WARNING: no policy specified for kudu/127.11.14.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.14.125@KRBTEST.COM" created.
Aug 07 18:35:48 dist-test-slave-6k22 krb5kdc[25671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127748, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.126@KRBTEST.COM for kudu/127.11.14.126@KRBTEST.COM
I20260807 18:35:48.798396 25766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:35:48.779377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:40395 (local address 127.0.0.1:43806)
0807 18:35:48.779959 (+ 582us) negotiation.cc:107] Waiting for socket to connect
0807 18:35:48.780016 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0807 18:35:48.780966 (+ 950us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:35:48.781874 (+ 908us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:35:48.781879 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:35:48.782235 (+ 356us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:35:48.782639 (+ 404us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:35:48.782697 (+ 58us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:35:48.784149 (+ 1452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:35:48.784153 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:35:48.784989 (+ 836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:35:48.784999 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:35:48.785754 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:35:48.785759 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:35:48.785862 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:35:48.786594 (+ 732us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:35:48.786614 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:35:48.792003 (+ 5389us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:35:48.794488 (+ 2485us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:35:48.794501 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:35:48.794516 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:35:48.794916 (+ 400us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:35:48.795397 (+ 481us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:35:48.795401 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:35:48.795402 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:35:48.795548 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:35:48.796750 (+ 1202us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:35:48.796759 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:35:48.796928 (+ 169us) client_negotiation.cc:769] Sending connection context
0807 18:35:48.797097 (+ 169us) client_negotiation.cc:240] Negotiation successful
0807 18:35:48.797429 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":349,"thread_start_us":197,"threads_started":1}
I20260807 18:35:48.798401 25767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:35:48.780379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43806 (local address 127.11.14.126:40395)
0807 18:35:48.780761 (+ 382us) server_negotiation.cc:206] Beginning negotiation
0807 18:35:48.780776 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 18:35:48.780830 (+ 54us) server_negotiation.cc:407] Connection header received
0807 18:35:48.781314 (+ 484us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:35:48.781324 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:35:48.781594 (+ 270us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:35:48.781863 (+ 269us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:35:48.782880 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:35:48.784000 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:35:48.785130 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:35:48.785565 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:35:48.785786 (+ 221us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:35:48.792246 (+ 6460us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:35:48.792259 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:35:48.792284 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:35:48.792326 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:35:48.794339 (+ 2013us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:35:48.795061 (+ 722us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:35:48.795071 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:35:48.795076 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:35:48.795255 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:35:48.795846 (+ 591us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:35:48.795853 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:35:48.795858 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:35:48.796563 (+ 705us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:35:48.796742 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0807 18:35:48.797255 (+ 513us) server_negotiation.cc:299] Negotiation successful
0807 18:35:48.797581 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":66,"server-negotiator.queue_time_us":217,"thread_start_us":97,"threads_started":1}
I20260807 18:35:48.801508 25763 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.14.125" port: 36793 } has no permanent_uuid. Determining permanent_uuid...
I20260807 18:35:48.802487 25766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:35:48.802085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.125:36793 (local address 127.0.0.1:51932)
0807 18:35:48.802222 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0807 18:35:48.802361 (+ 139us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.125:36793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260807 18:35:48.802758 25707 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.11.14.125:36793: connect: Connection refused (error 111)
I20260807 18:35:48.803874 25766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:35:48.803489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.125:36793 (local address 127.0.0.1:51942)
0807 18:35:48.803752 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0807 18:35:48.803813 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.125:36793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126}
W20260807 18:35:48.804159 25763 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.14.125:36793: Network error: Client connection negotiation failed: client connection to 127.11.14.125:36793: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.14.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.125.keytab.
Entry for principal kudu/127.11.14.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.125.keytab.
I20260807 18:35:48.811161 11321 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkz56XN/build/debug/bin/kudu
/tmp/dist-test-taskkz56XN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786127579900523-11321-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786127579900523-11321-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786127579900523-11321-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786127579900523-11321-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.14.125:36793
--webserver_interface=127.11.14.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.125.keytab
--principal=kudu/127.11.14.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.14.84:34377
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.14.126:40395,127.11.14.125:36793,127.11.14.124:37595
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786127579900523-11321-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786127579900523-11321-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786127579900523-11321-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 18:35:48.853541 25763 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.14.125" port: 36793 } attempt: 1
I20260807 18:35:48.854605 25766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:35:48.854223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.125:36793 (local address 127.0.0.1:51956)
0807 18:35:48.854438 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0807 18:35:48.854523 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.125:36793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90}
I20260807 18:35:48.855427 25766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:35:48.855189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.125:36793 (local address 127.0.0.1:51962)
0807 18:35:48.855317 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0807 18:35:48.855369 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.125:36793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260807 18:35:48.855553 25763 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.14.125:36793: Network error: Client connection negotiation failed: client connection to 127.11.14.125:36793: connect: Connection refused (error 111)
I20260807 18:35:48.923733 25763 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.14.125" port: 36793 } attempt: 2
I20260807 18:35:48.924644 25766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:35:48.924324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.125:36793 (local address 127.0.0.1:51968)
0807 18:35:48.924498 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0807 18:35:48.924567 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.125:36793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260807 18:35:48.925400 25766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:35:48.925156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.125:36793 (local address 127.0.0.1:51972)
0807 18:35:48.925280 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0807 18:35:48.925337 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.125:36793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260807 18:35:48.925540 25763 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.14.125:36793: Network error: Client connection negotiation failed: client connection to 127.11.14.125:36793: connect: Connection refused (error 111)
W20260807 18:35:48.939198 25770 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:35:48.939659 25770 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:35:48.939810 25770 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:35:48.944525 25770 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:35:48.944657 25770 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:35:48.944705 25770 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:35:48.944749 25770 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:35:48.944790 25770 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:35:48.950939 25770 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.14.84:34377
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786127579900523-11321-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786127579900523-11321-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.11.14.126:40395,127.11.14.125:36793,127.11.14.124:37595
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.125.keytab
--principal=kudu/127.11.14.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.14.125:36793
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786127579900523-11321-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.11.14.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25770
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786127579900523-11321-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:35:48.952270 25770 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:35:48.953378 25770 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:35:48.962524 25778 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:35:48.962543 25775 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:35:48.962576 25776 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:35:48.962594 25770 server_base.cc:1034] running on GCE node
I20260807 18:35:48.963356 25770 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:35:48.964521 25770 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:35:48.965716 25770 hybrid_clock.cc:648] HybridClock initialized: now 1786127748965686 us; error 49 us; skew 500 ppm
Aug 07 18:35:48 dist-test-slave-6k22 krb5kdc[25671](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127748, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:35:48.968706 25770 init.cc:377] Logged in from keytab as kudu/127.11.14.125@KRBTEST.COM (short username kudu)
I20260807 18:35:48.969621 25770 webserver.cc:460] Webserver started at http://127.11.14.125:34981/ using document root <none> and password file <none>
I20260807 18:35:48.970237 25770 fs_manager.cc:359] Metadata directory not provided
I20260807 18:35:48.970335 25770 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:35:48.970583 25770 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:35:48.972776 25770 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786127579900523-11321-0/minicluster-data/master-1/data/instance:
uuid: "4e83dc1132fe48908f77dcc7e200f946"
format_stamp: "Formatted at 2026-08-07 18:35:48 on dist-test-slave-6k22"
server_key: "6a62cd464c25b06f96c5cec948434196"
server_key_iv: "535721582d9ecba94ec460bc21a183f3"
server_key_version: "encryptionkey@0"
I20260807 18:35:48.973379 25770 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786127579900523-11321-0/minicluster-data/master-1/wal/instance:
uuid: "4e83dc1132fe48908f77dcc7e200f946"
format_stamp: "Formatted at 2026-08-07 18:35:48 on dist-test-slave-6k22"
server_key: "6a62cd464c25b06f96c5cec948434196"
server_key_iv: "535721582d9ecba94ec460bc21a183f3"
server_key_version: "encryptionkey@0"
I20260807 18:35:48.977305 25770 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 18:35:48.979847 25785 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:35:48.981101 25770 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260807 18:35:48.981256 25770 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786127579900523-11321-0/minicluster-data/master-1/data,/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786127579900523-11321-0/minicluster-data/master-1/wal
uuid: "4e83dc1132fe48908f77dcc7e200f946"
format_stamp: "Formatted at 2026-08-07 18:35:48 on dist-test-slave-6k22"
server_key: "6a62cd464c25b06f96c5cec948434196"
server_key_iv: "535721582d9ecba94ec460bc21a183f3"
server_key_version: "encryptionkey@0"
I20260807 18:35:48.981381 25770 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786127579900523-11321-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786127579900523-11321-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786127579900523-11321-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:35:48.990803 25770 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:35:48.993855 25770 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:35:48.994057 25770 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:35:49.001116 25770 rpc_server.cc:307] RPC server started. Bound to: 127.11.14.125:36793
I20260807 18:35:49.001176 25844 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.14.125:36793 every 8 connection(s)
I20260807 18:35:49.002115 25770 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786127579900523-11321-0/minicluster-data/master-1/data/info.pb
I20260807 18:35:49.005064 25845 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:35:49.009378 11321 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkz56XN/build/debug/bin/kudu as pid 25770
I20260807 18:35:49.009014 25845 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.14.126" port: 40395 } has no permanent_uuid. Determining permanent_uuid...
I20260807 18:35:49.009519 11321 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786127579900523-11321-0/minicluster-data/master-1/wal/instance
I20260807 18:35:49.009836 11321 external_mini_cluster.cc:1428] Setting key 4048e76c660f9a45bcefe4e362696bbc
I20260807 18:35:49.016709 25763 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.14.125" port: 36793 } attempt: 3
WARNING: no policy specified for kudu/127.11.14.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.14.124@KRBTEST.COM" created.
Aug 07 18:35:49 dist-test-slave-6k22 krb5kdc[25671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127748, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.125@KRBTEST.COM for kudu/127.11.14.126@KRBTEST.COM
Aug 07 18:35:49 dist-test-slave-6k22 krb5kdc[25671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127748, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.126@KRBTEST.COM for kudu/127.11.14.125@KRBTEST.COM
I20260807 18:35:49.030771 25766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:35:49.017181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.125:36793 (local address 127.0.0.1:51974)
0807 18:35:49.017318 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0807 18:35:49.017334 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 18:35:49.017416 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:35:49.017928 (+ 512us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:35:49.017931 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:35:49.018129 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:35:49.018313 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:35:49.018319 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:35:49.019892 (+ 1573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:35:49.019896 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:35:49.020684 (+ 788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:35:49.020696 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:35:49.021314 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:35:49.021318 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:35:49.021434 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:35:49.022024 (+ 590us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:35:49.022036 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:35:49.026010 (+ 3974us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:35:49.028678 (+ 2668us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:35:49.028681 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:35:49.028683 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:35:49.028891 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:35:49.029300 (+ 409us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:35:49.029303 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:35:49.029305 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:35:49.029445 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:35:49.030184 (+ 739us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:35:49.030188 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:35:49.030296 (+ 108us) client_negotiation.cc:769] Sending connection context
0807 18:35:49.030421 (+ 125us) client_negotiation.cc:240] Negotiation successful
0807 18:35:49.030582 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44,"mutex_wait_us":44}
I20260807 18:35:49.031193 25767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:35:49.011387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43822 (local address 127.11.14.126:40395)
0807 18:35:49.011585 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0807 18:35:49.011591 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:35:49.012601 (+ 1010us) server_negotiation.cc:407] Connection header received
0807 18:35:49.013896 (+ 1295us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:35:49.013899 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:35:49.013960 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:35:49.014037 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:35:49.015765 (+ 1728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:35:49.016286 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:35:49.018189 (+ 1903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:35:49.018562 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:35:49.018678 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:35:49.025886 (+ 7208us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:35:49.025894 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:35:49.025900 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:35:49.025938 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:35:49.028418 (+ 2480us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:35:49.029068 (+ 650us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:35:49.029074 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:35:49.029079 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:35:49.029158 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:35:49.029580 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:35:49.029586 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:35:49.029591 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:35:49.029875 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:35:49.029969 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0807 18:35:49.030592 (+ 623us) server_negotiation.cc:299] Negotiation successful
0807 18:35:49.030748 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":62}
I20260807 18:35:49.031985 25850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:35:49.017259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51974 (local address 127.11.14.125:36793)
0807 18:35:49.017558 (+ 299us) server_negotiation.cc:206] Beginning negotiation
0807 18:35:49.017568 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 18:35:49.017590 (+ 22us) server_negotiation.cc:407] Connection header received
0807 18:35:49.017626 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:35:49.017630 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:35:49.017775 (+ 145us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:35:49.017943 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:35:49.018456 (+ 513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:35:49.019754 (+ 1298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:35:49.020815 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:35:49.021185 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:35:49.021310 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:35:49.026171 (+ 4861us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:35:49.026178 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:35:49.026192 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:35:49.026214 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:35:49.028555 (+ 2341us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:35:49.029017 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:35:49.029024 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:35:49.029026 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:35:49.029181 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:35:49.029601 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:35:49.029607 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:35:49.029612 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:35:49.030053 (+ 441us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:35:49.030211 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0807 18:35:49.030530 (+ 319us) server_negotiation.cc:299] Negotiation successful
0807 18:35:49.030884 (+ 354us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":191,"thread_start_us":111,"threads_started":1}
I20260807 18:35:49.032011 25849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:35:49.011793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:40395 (local address 127.0.0.1:43822)
0807 18:35:49.012398 (+ 605us) negotiation.cc:107] Waiting for socket to connect
0807 18:35:49.012464 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0807 18:35:49.013675 (+ 1211us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:35:49.014257 (+ 582us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:35:49.014269 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:35:49.014771 (+ 502us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:35:49.015592 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:35:49.015620 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:35:49.016434 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:35:49.016440 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:35:49.017999 (+ 1559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:35:49.018015 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:35:49.018719 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:35:49.018723 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:35:49.018950 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:35:49.019628 (+ 678us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:35:49.019654 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:35:49.025649 (+ 5995us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:35:49.028553 (+ 2904us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:35:49.028564 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:35:49.028577 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:35:49.028931 (+ 354us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:35:49.029266 (+ 335us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:35:49.029272 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:35:49.029277 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:35:49.029453 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:35:49.030048 (+ 595us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:35:49.030058 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:35:49.030244 (+ 186us) client_negotiation.cc:769] Sending connection context
0807 18:35:49.031393 (+ 1149us) client_negotiation.cc:240] Negotiation successful
0807 18:35:49.031649 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":321,"thread_start_us":118,"threads_started":1}
I20260807 18:35:49.033999 25845 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.14.125" port: 36793 } has no permanent_uuid. Determining permanent_uuid...
I20260807 18:35:49.034232 25763 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.14.124" port: 37595 } has no permanent_uuid. Determining permanent_uuid...
I20260807 18:35:49.035058 25766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:35:49.034700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.124:37595 (local address 127.0.0.1:49498)
0807 18:35:49.034906 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0807 18:35:49.034970 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.124:37595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117}
I20260807 18:35:49.035888 25766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:35:49.035634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.124:37595 (local address 127.0.0.1:49506)
0807 18:35:49.035779 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0807 18:35:49.035827 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.124:37595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
W20260807 18:35:49.036098 25763 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.14.124:37595: Network error: Client connection negotiation failed: client connection to 127.11.14.124:37595: connect: Connection refused (error 111)
Aug 07 18:35:49 dist-test-slave-6k22 krb5kdc[25671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127748, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.125@KRBTEST.COM for kudu/127.11.14.125@KRBTEST.COM
I20260807 18:35:49.042481 25849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:35:49.034574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.125:36793 (local address 127.0.0.1:51978)
0807 18:35:49.034691 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0807 18:35:49.034707 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 18:35:49.034787 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:35:49.035020 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:35:49.035023 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:35:49.035195 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:35:49.035311 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:35:49.035319 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:35:49.036402 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:35:49.036405 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:35:49.036945 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:35:49.036953 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:35:49.037463 (+ 510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:35:49.037466 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:35:49.037551 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:35:49.037968 (+ 417us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:35:49.037979 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:35:49.039232 (+ 1253us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:35:49.041020 (+ 1788us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:35:49.041025 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:35:49.041027 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:35:49.041217 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:35:49.041499 (+ 282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:35:49.041502 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:35:49.041504 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:35:49.041585 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:35:49.041955 (+ 370us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:35:49.041958 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:35:49.042040 (+ 82us) client_negotiation.cc:769] Sending connection context
0807 18:35:49.042154 (+ 114us) client_negotiation.cc:240] Negotiation successful
0807 18:35:49.042293 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47}
I20260807 18:35:49.042481 25850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:35:49.034734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51978 (local address 127.11.14.125:36793)
0807 18:35:49.034828 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0807 18:35:49.034844 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 18:35:49.034857 (+ 13us) server_negotiation.cc:407] Connection header received
0807 18:35:49.034886 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:35:49.034888 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:35:49.034928 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:35:49.035009 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:35:49.035457 (+ 448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:35:49.036273 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:35:49.037051 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:35:49.037334 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:35:49.037460 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:35:49.039362 (+ 1902us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:35:49.039365 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:35:49.039367 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:35:49.039388 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:35:49.040889 (+ 1501us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:35:49.041316 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:35:49.041321 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:35:49.041322 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:35:49.041400 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:35:49.041674 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:35:49.041678 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:35:49.041680 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:35:49.041858 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:35:49.041965 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0807 18:35:49.042199 (+ 234us) server_negotiation.cc:299] Negotiation successful
0807 18:35:49.042351 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":54,"server-negotiator.queue_time_us":34}
I20260807 18:35:49.043830 25845 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.14.124" port: 37595 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.14.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.124.keytab.
Entry for principal kudu/127.11.14.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.124.keytab.
I20260807 18:35:49.044615 25849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:35:49.044340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.124:37595 (local address 127.0.0.1:49518)
0807 18:35:49.044460 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0807 18:35:49.044542 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.124:37595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260807 18:35:49.044929 25787 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.11.14.124:37595: connect: Connection refused (error 111)
I20260807 18:35:49.045455 11321 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkz56XN/build/debug/bin/kudu
/tmp/dist-test-taskkz56XN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786127579900523-11321-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786127579900523-11321-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786127579900523-11321-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786127579900523-11321-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.14.124:37595
--webserver_interface=127.11.14.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.124.keytab
--principal=kudu/127.11.14.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.14.84:34377
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.14.126:40395,127.11.14.125:36793,127.11.14.124:37595
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786127579900523-11321-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786127579900523-11321-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786127579900523-11321-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 18:35:49.045929 25849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:35:49.045691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.124:37595 (local address 127.0.0.1:49526)
0807 18:35:49.045812 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0807 18:35:49.045877 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.124:37595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260807 18:35:49.046077 25845 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.14.124:37595: Network error: Client connection negotiation failed: client connection to 127.11.14.124:37595: connect: Connection refused (error 111)
I20260807 18:35:49.067273 25763 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.14.124" port: 37595 } attempt: 1
I20260807 18:35:49.068305 25766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:35:49.067980 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.124:37595 (local address 127.0.0.1:49536)
0807 18:35:49.068151 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0807 18:35:49.068219 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.124:37595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260807 18:35:49.069084 25766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:35:49.068876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.124:37595 (local address 127.0.0.1:49552)
0807 18:35:49.068985 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0807 18:35:49.069029 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.124:37595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260807 18:35:49.069263 25763 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.14.124:37595: Network error: Client connection negotiation failed: client connection to 127.11.14.124:37595: connect: Connection refused (error 111)
I20260807 18:35:49.095366 25845 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.14.124" port: 37595 } attempt: 1
I20260807 18:35:49.096315 25849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:35:49.096000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.124:37595 (local address 127.0.0.1:49562)
0807 18:35:49.096160 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0807 18:35:49.096232 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.124:37595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260807 18:35:49.097147 25849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:35:49.096923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.124:37595 (local address 127.0.0.1:49578)
0807 18:35:49.097041 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0807 18:35:49.097093 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.124:37595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260807 18:35:49.097322 25845 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.14.124:37595: Network error: Client connection negotiation failed: client connection to 127.11.14.124:37595: connect: Connection refused (error 111)
I20260807 18:35:49.144441 25763 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.14.124" port: 37595 } attempt: 2
I20260807 18:35:49.145447 25766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:35:49.145113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.124:37595 (local address 127.0.0.1:49592)
0807 18:35:49.145280 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0807 18:35:49.145365 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.124:37595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260807 18:35:49.146219 25766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:35:49.146013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.124:37595 (local address 127.0.0.1:49598)
0807 18:35:49.146119 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0807 18:35:49.146164 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.124:37595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260807 18:35:49.146371 25763 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.14.124:37595: Network error: Client connection negotiation failed: client connection to 127.11.14.124:37595: connect: Connection refused (error 111)
I20260807 18:35:49.165508 25845 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.14.124" port: 37595 } attempt: 2
I20260807 18:35:49.166368 25849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:35:49.166064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.124:37595 (local address 127.0.0.1:49614)
0807 18:35:49.166211 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0807 18:35:49.166281 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.124:37595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260807 18:35:49.167176 25849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:35:49.166951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.124:37595 (local address 127.0.0.1:49626)
0807 18:35:49.167066 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0807 18:35:49.167120 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.124:37595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260807 18:35:49.167362 25845 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.14.124:37595: Network error: Client connection negotiation failed: client connection to 127.11.14.124:37595: connect: Connection refused (error 111)
W20260807 18:35:49.167238 25854 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:35:49.167626 25854 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:35:49.167683 25854 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:35:49.171736 25854 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:35:49.171793 25854 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:35:49.171813 25854 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:35:49.171830 25854 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:35:49.171845 25854 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:35:49.177248 25854 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.14.84:34377
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786127579900523-11321-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786127579900523-11321-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.11.14.126:40395,127.11.14.125:36793,127.11.14.124:37595
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.124.keytab
--principal=kudu/127.11.14.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.14.124:37595
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786127579900523-11321-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.11.14.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25854
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786127579900523-11321-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:35:49.178241 25854 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:35:49.179235 25854 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:35:49.188179 25862 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:35:49.188189 25860 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:35:49.188212 25859 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:35:49.188354 25854 server_base.cc:1034] running on GCE node
I20260807 18:35:49.188977 25854 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:35:49.190188 25854 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:35:49.191398 25854 hybrid_clock.cc:648] HybridClock initialized: now 1786127749191383 us; error 26 us; skew 500 ppm
Aug 07 18:35:49 dist-test-slave-6k22 krb5kdc[25671](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127749, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:35:49.194409 25854 init.cc:377] Logged in from keytab as kudu/127.11.14.124@KRBTEST.COM (short username kudu)
I20260807 18:35:49.195341 25854 webserver.cc:460] Webserver started at http://127.11.14.124:38953/ using document root <none> and password file <none>
I20260807 18:35:49.195995 25854 fs_manager.cc:359] Metadata directory not provided
I20260807 18:35:49.196096 25854 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:35:49.196352 25854 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:35:49.198472 25854 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786127579900523-11321-0/minicluster-data/master-2/data/instance:
uuid: "9516f04f2b484fcba774e527313b028d"
format_stamp: "Formatted at 2026-08-07 18:35:49 on dist-test-slave-6k22"
server_key: "461ef7852bd2bdcbd6579347f03a006e"
server_key_iv: "39025af5876593ad7b435d82b069d01a"
server_key_version: "encryptionkey@0"
I20260807 18:35:49.199100 25854 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786127579900523-11321-0/minicluster-data/master-2/wal/instance:
uuid: "9516f04f2b484fcba774e527313b028d"
format_stamp: "Formatted at 2026-08-07 18:35:49 on dist-test-slave-6k22"
server_key: "461ef7852bd2bdcbd6579347f03a006e"
server_key_iv: "39025af5876593ad7b435d82b069d01a"
server_key_version: "encryptionkey@0"
I20260807 18:35:49.203342 25854 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260807 18:35:49.206192 25871 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:35:49.207718 25854 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:35:49.207868 25854 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786127579900523-11321-0/minicluster-data/master-2/data,/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786127579900523-11321-0/minicluster-data/master-2/wal
uuid: "9516f04f2b484fcba774e527313b028d"
format_stamp: "Formatted at 2026-08-07 18:35:49 on dist-test-slave-6k22"
server_key: "461ef7852bd2bdcbd6579347f03a006e"
server_key_iv: "39025af5876593ad7b435d82b069d01a"
server_key_version: "encryptionkey@0"
I20260807 18:35:49.208004 25854 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786127579900523-11321-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786127579900523-11321-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786127579900523-11321-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:35:49.229583 25854 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:35:49.232671 25854 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:35:49.232910 25854 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:35:49.240433 25854 rpc_server.cc:307] RPC server started. Bound to: 127.11.14.124:37595
I20260807 18:35:49.240449 25926 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.14.124:37595 every 8 connection(s)
I20260807 18:35:49.241480 25854 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786127579900523-11321-0/minicluster-data/master-2/data/info.pb
I20260807 18:35:49.243520 11321 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkz56XN/build/debug/bin/kudu as pid 25854
I20260807 18:35:49.243703 11321 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786127579900523-11321-0/minicluster-data/master-2/wal/instance
I20260807 18:35:49.244024 11321 external_mini_cluster.cc:1428] Setting key 6c34ddaf01f897e1fc7db96dda102a44
I20260807 18:35:49.244659 25927 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:35:49.245543 25763 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.14.124" port: 37595 } attempt: 3
I20260807 18:35:49.250242 25927 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.14.126" port: 40395 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 18:35:49 dist-test-slave-6k22 krb5kdc[25671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127748, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.126@KRBTEST.COM for kudu/127.11.14.124@KRBTEST.COM
Aug 07 18:35:49 dist-test-slave-6k22 krb5kdc[25671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127748, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.14.124@KRBTEST.COM
I20260807 18:35:49.258527 25845 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.14.124" port: 37595 } attempt: 3
I20260807 18:35:49.258744 25766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:35:49.246070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.124:37595 (local address 127.0.0.1:49644)
0807 18:35:49.246213 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0807 18:35:49.246228 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 18:35:49.246314 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:35:49.247596 (+ 1282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:35:49.247600 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:35:49.247818 (+ 218us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:35:49.247951 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:35:49.247959 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:35:49.249612 (+ 1653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:35:49.249616 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:35:49.250456 (+ 840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:35:49.250469 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:35:49.251340 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:35:49.251344 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:35:49.251432 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:35:49.252138 (+ 706us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:35:49.252160 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:35:49.253763 (+ 1603us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:35:49.256717 (+ 2954us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:35:49.256721 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:35:49.256723 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:35:49.257005 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:35:49.257392 (+ 387us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:35:49.257397 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:35:49.257401 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:35:49.257505 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:35:49.258187 (+ 682us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:35:49.258193 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:35:49.258306 (+ 113us) client_negotiation.cc:769] Sending connection context
0807 18:35:49.258437 (+ 131us) client_negotiation.cc:240] Negotiation successful
0807 18:35:49.258581 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55}
Aug 07 18:35:49 dist-test-slave-6k22 krb5kdc[25671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127749, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.124@KRBTEST.COM for kudu/127.11.14.126@KRBTEST.COM
I20260807 18:35:49.260046 25930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:35:49.245535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49638 (local address 127.11.14.124:37595)
0807 18:35:49.246033 (+ 498us) server_negotiation.cc:206] Beginning negotiation
0807 18:35:49.246056 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0807 18:35:49.246100 (+ 44us) server_negotiation.cc:407] Connection header received
0807 18:35:49.246875 (+ 775us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:35:49.246893 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:35:49.247249 (+ 356us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:35:49.247642 (+ 393us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:35:49.248471 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:35:49.249377 (+ 906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:35:49.250622 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:35:49.251172 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:35:49.251376 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:35:49.254691 (+ 3315us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:35:49.254694 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:35:49.254697 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:35:49.254726 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:35:49.256795 (+ 2069us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:35:49.257266 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:35:49.257271 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:35:49.257273 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:35:49.257380 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:35:49.257703 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:35:49.257708 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:35:49.257711 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:35:49.258069 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:35:49.258300 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0807 18:35:49.258623 (+ 323us) server_negotiation.cc:299] Negotiation successful
0807 18:35:49.258959 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":5,"server-negotiator.queue_time_us":309,"spinlock_wait_cycles":6912,"thread_start_us":120,"threads_started":1}
I20260807 18:35:49.260061 25931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:35:49.246107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49644 (local address 127.11.14.124:37595)
0807 18:35:49.246415 (+ 308us) server_negotiation.cc:206] Beginning negotiation
0807 18:35:49.246424 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:35:49.246440 (+ 16us) server_negotiation.cc:407] Connection header received
0807 18:35:49.246897 (+ 457us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:35:49.246902 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:35:49.247253 (+ 351us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:35:49.248075 (+ 822us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:35:49.248221 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:35:49.249432 (+ 1211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:35:49.250606 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:35:49.251175 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:35:49.251696 (+ 521us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:35:49.253960 (+ 2264us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:35:49.253966 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:35:49.253976 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:35:49.254001 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:35:49.256570 (+ 2569us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:35:49.257132 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:35:49.257138 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:35:49.257145 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:35:49.257276 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:35:49.257608 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:35:49.257612 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:35:49.257614 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:35:49.258067 (+ 453us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:35:49.258227 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0807 18:35:49.259437 (+ 1210us) server_negotiation.cc:299] Negotiation successful
0807 18:35:49.259564 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":74,"server-negotiator.queue_time_us":241,"spinlock_wait_cycles":4352,"thread_start_us":129,"threads_started":1}
Aug 07 18:35:49 dist-test-slave-6k22 krb5kdc[25671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127748, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.125@KRBTEST.COM for kudu/127.11.14.124@KRBTEST.COM
I20260807 18:35:49.265585 25932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:35:49.252366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:40395 (local address 127.0.0.1:43838)
0807 18:35:49.252824 (+ 458us) negotiation.cc:107] Waiting for socket to connect
0807 18:35:49.252882 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0807 18:35:49.253047 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:35:49.253492 (+ 445us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:35:49.253499 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:35:49.253972 (+ 473us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:35:49.254155 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:35:49.254165 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:35:49.255151 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:35:49.255156 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:35:49.256113 (+ 957us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:35:49.256124 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:35:49.256604 (+ 480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:35:49.256607 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:35:49.256732 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:35:49.258050 (+ 1318us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:35:49.258063 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:35:49.259960 (+ 1897us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:35:49.263525 (+ 3565us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:35:49.263535 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:35:49.263540 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:35:49.263901 (+ 361us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:35:49.264192 (+ 291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:35:49.264195 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:35:49.264197 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:35:49.264286 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:35:49.264853 (+ 567us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:35:49.264859 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:35:49.265002 (+ 143us) client_negotiation.cc:769] Sending connection context
0807 18:35:49.265140 (+ 138us) client_negotiation.cc:240] Negotiation successful
0807 18:35:49.265373 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":123,"threads_started":1}
I20260807 18:35:49.266076 25767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:35:49.252413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43838 (local address 127.11.14.126:40395)
0807 18:35:49.252530 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0807 18:35:49.252535 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:35:49.252959 (+ 424us) server_negotiation.cc:407] Connection header received
0807 18:35:49.253166 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:35:49.253172 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:35:49.253227 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:35:49.253278 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:35:49.254292 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:35:49.255023 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:35:49.256243 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:35:49.256481 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:35:49.256599 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:35:49.260112 (+ 3513us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:35:49.260116 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:35:49.260119 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:35:49.260145 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:35:49.263414 (+ 3269us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:35:49.263984 (+ 570us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:35:49.263988 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:35:49.263990 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:35:49.264081 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:35:49.264396 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:35:49.264399 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:35:49.264401 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:35:49.264688 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:35:49.264794 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0807 18:35:49.265804 (+ 1010us) server_negotiation.cc:299] Negotiation successful
0807 18:35:49.265923 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260807 18:35:49.266048 25763 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5532e9367168440c949f3e3a203cef06: Bootstrap starting.
I20260807 18:35:49.269213 25763 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5532e9367168440c949f3e3a203cef06: Neither blocks nor log segments found. Creating new log.
I20260807 18:35:49.270151 25763 log.cc:824] T 00000000000000000000000000000000 P 5532e9367168440c949f3e3a203cef06: Log is configured to *not* fsync() on all Append() calls
Aug 07 18:35:49 dist-test-slave-6k22 krb5kdc[25671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127748, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.14.126@KRBTEST.COM
I20260807 18:35:49.270516 25849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:35:49.259097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.124:37595 (local address 127.0.0.1:49656)
0807 18:35:49.259225 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0807 18:35:49.259242 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 18:35:49.259328 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:35:49.260762 (+ 1434us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:35:49.260766 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:35:49.260966 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:35:49.261111 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:35:49.261122 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:35:49.262407 (+ 1285us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:35:49.262412 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:35:49.263156 (+ 744us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:35:49.263169 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:35:49.263838 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:35:49.263842 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:35:49.263938 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:35:49.264431 (+ 493us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:35:49.264446 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:35:49.266413 (+ 1967us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:35:49.268472 (+ 2059us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:35:49.268476 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:35:49.268478 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:35:49.268999 (+ 521us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:35:49.269383 (+ 384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:35:49.269387 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:35:49.269389 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:35:49.269501 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:35:49.269962 (+ 461us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:35:49.269973 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:35:49.270077 (+ 104us) client_negotiation.cc:769] Sending connection context
0807 18:35:49.270193 (+ 116us) client_negotiation.cc:240] Negotiation successful
0807 18:35:49.270347 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53}
I20260807 18:35:49.270543 25933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:35:49.259637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49656 (local address 127.11.14.124:37595)
0807 18:35:49.260232 (+ 595us) server_negotiation.cc:206] Beginning negotiation
0807 18:35:49.260245 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 18:35:49.260259 (+ 14us) server_negotiation.cc:407] Connection header received
0807 18:35:49.260296 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:35:49.260300 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:35:49.260350 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:35:49.261484 (+ 1134us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:35:49.261588 (+ 104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:35:49.262299 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:35:49.263301 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:35:49.263701 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:35:49.263820 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:35:49.266561 (+ 2741us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:35:49.266565 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:35:49.266567 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:35:49.266591 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:35:49.268351 (+ 1760us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:35:49.269134 (+ 783us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:35:49.269140 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:35:49.269142 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:35:49.269232 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:35:49.269612 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:35:49.269616 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:35:49.269618 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:35:49.269857 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:35:49.269952 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0807 18:35:49.270258 (+ 306us) server_negotiation.cc:299] Negotiation successful
0807 18:35:49.270382 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":526,"thread_start_us":109,"threads_started":1}
I20260807 18:35:49.272097 25927 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.14.125" port: 36793 } has no permanent_uuid. Determining permanent_uuid...
I20260807 18:35:49.273345 25763 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5532e9367168440c949f3e3a203cef06: No bootstrap required, opened a new log
I20260807 18:35:49.274987 25845 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4e83dc1132fe48908f77dcc7e200f946: Bootstrap starting.
I20260807 18:35:49.275434 25934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:35:49.264451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43844 (local address 127.11.14.126:40395)
0807 18:35:49.264777 (+ 326us) server_negotiation.cc:206] Beginning negotiation
0807 18:35:49.264790 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 18:35:49.264803 (+ 13us) server_negotiation.cc:407] Connection header received
0807 18:35:49.264848 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:35:49.264851 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:35:49.264894 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:35:49.265757 (+ 863us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:35:49.265882 (+ 125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:35:49.266653 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:35:49.267874 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:35:49.268295 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:35:49.268715 (+ 420us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:35:49.271326 (+ 2611us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:35:49.271329 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:35:49.271332 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:35:49.271358 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:35:49.273132 (+ 1774us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:35:49.273872 (+ 740us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:35:49.273877 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:35:49.273879 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:35:49.273971 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:35:49.274293 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:35:49.274296 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:35:49.274298 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:35:49.274523 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:35:49.274740 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0807 18:35:49.274940 (+ 200us) server_negotiation.cc:299] Negotiation successful
0807 18:35:49.275067 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":229,"thread_start_us":78,"threads_started":1}
I20260807 18:35:49.277446 25763 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5532e9367168440c949f3e3a203cef06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5532e9367168440c949f3e3a203cef06" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 40395 } } peers { permanent_uuid: "4e83dc1132fe48908f77dcc7e200f946" member_type: VOTER last_known_addr { host: "127.11.14.125" port: 36793 } } peers { permanent_uuid: "9516f04f2b484fcba774e527313b028d" member_type: VOTER last_known_addr { host: "127.11.14.124" port: 37595 } }
I20260807 18:35:49.277698 25763 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5532e9367168440c949f3e3a203cef06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:35:49.277789 25763 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5532e9367168440c949f3e3a203cef06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5532e9367168440c949f3e3a203cef06, State: Initialized, Role: FOLLOWER
I20260807 18:35:49.278085 25845 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4e83dc1132fe48908f77dcc7e200f946: Neither blocks nor log segments found. Creating new log.
I20260807 18:35:49.278267 25763 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5532e9367168440c949f3e3a203cef06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5532e9367168440c949f3e3a203cef06" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 40395 } } peers { permanent_uuid: "4e83dc1132fe48908f77dcc7e200f946" member_type: VOTER last_known_addr { host: "127.11.14.125" port: 36793 } } peers { permanent_uuid: "9516f04f2b484fcba774e527313b028d" member_type: VOTER last_known_addr { host: "127.11.14.124" port: 37595 } }
I20260807 18:35:49.278774 25937 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5532e9367168440c949f3e3a203cef06 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5532e9367168440c949f3e3a203cef06" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 40395 } } peers { permanent_uuid: "4e83dc1132fe48908f77dcc7e200f946" member_type: VOTER last_known_addr { host: "127.11.14.125" port: 36793 } } peers { permanent_uuid: "9516f04f2b484fcba774e527313b028d" member_type: VOTER last_known_addr { host: "127.11.14.124" port: 37595 } } }
I20260807 18:35:49.278913 25937 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5532e9367168440c949f3e3a203cef06 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 18:35:49.279033 25845 log.cc:824] T 00000000000000000000000000000000 P 4e83dc1132fe48908f77dcc7e200f946: Log is configured to *not* fsync() on all Append() calls
I20260807 18:35:49.279754 25763 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5532e9367168440c949f3e3a203cef06 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:35:49.281548 25845 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4e83dc1132fe48908f77dcc7e200f946: No bootstrap required, opened a new log
Aug 07 18:35:49 dist-test-slave-6k22 krb5kdc[25671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127748, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.14.125@KRBTEST.COM
W20260807 18:35:49.286062 25949 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5532e9367168440c949f3e3a203cef06: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 18:35:49.286141 25949 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 18:35:49.286496 25845 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4e83dc1132fe48908f77dcc7e200f946 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5532e9367168440c949f3e3a203cef06" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 40395 } } peers { permanent_uuid: "4e83dc1132fe48908f77dcc7e200f946" member_type: VOTER last_known_addr { host: "127.11.14.125" port: 36793 } } peers { permanent_uuid: "9516f04f2b484fcba774e527313b028d" member_type: VOTER last_known_addr { host: "127.11.14.124" port: 37595 } }
I20260807 18:35:49.287091 25845 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4e83dc1132fe48908f77dcc7e200f946 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:35:49.287191 25845 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4e83dc1132fe48908f77dcc7e200f946 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e83dc1132fe48908f77dcc7e200f946, State: Initialized, Role: FOLLOWER
I20260807 18:35:49.287986 25845 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4e83dc1132fe48908f77dcc7e200f946 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5532e9367168440c949f3e3a203cef06" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 40395 } } peers { permanent_uuid: "4e83dc1132fe48908f77dcc7e200f946" member_type: VOTER last_known_addr { host: "127.11.14.125" port: 36793 } } peers { permanent_uuid: "9516f04f2b484fcba774e527313b028d" member_type: VOTER last_known_addr { host: "127.11.14.124" port: 37595 } }
I20260807 18:35:49.289381 25845 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4e83dc1132fe48908f77dcc7e200f946 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:35:49.285892 25686 master_runner.cc:186] Error getting master registration for 127.11.14.125:36793: OK, instance_id { permanent_uuid: "4e83dc1132fe48908f77dcc7e200f946" instance_seqno: 1786127748999115 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 18:35:49.289748 25686 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 18:35:49.291070 25936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:35:49.276736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51984 (local address 127.11.14.125:36793)
0807 18:35:49.279895 (+ 3159us) server_negotiation.cc:206] Beginning negotiation
0807 18:35:49.279904 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:35:49.279916 (+ 12us) server_negotiation.cc:407] Connection header received
0807 18:35:49.279940 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:35:49.279943 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:35:49.279980 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:35:49.280832 (+ 852us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:35:49.280948 (+ 116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:35:49.281713 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:35:49.282776 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:35:49.283152 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:35:49.283660 (+ 508us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:35:49.286363 (+ 2703us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:35:49.286367 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:35:49.286370 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:35:49.286396 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:35:49.288388 (+ 1992us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:35:49.288912 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:35:49.288916 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:35:49.288917 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:35:49.289004 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:35:49.289317 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:35:49.289320 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:35:49.289322 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:35:49.289647 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:35:49.290633 (+ 986us) server_negotiation.cc:1035] Waiting for connection context
0807 18:35:49.290716 (+ 83us) server_negotiation.cc:299] Negotiation successful
0807 18:35:49.290867 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3022,"thread_start_us":117,"threads_started":1}
Aug 07 18:35:49 dist-test-slave-6k22 krb5kdc[25671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127749, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.124@KRBTEST.COM for kudu/127.11.14.125@KRBTEST.COM
I20260807 18:35:49.294651 11321 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 18:35:49.299203 25932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:35:49.272922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.125:36793 (local address 127.0.0.1:51982)
0807 18:35:49.279612 (+ 6690us) negotiation.cc:107] Waiting for socket to connect
0807 18:35:49.279631 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 18:35:49.279738 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:35:49.280158 (+ 420us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:35:49.280163 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:35:49.280389 (+ 226us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:35:49.280533 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:35:49.280545 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:35:49.281671 (+ 1126us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:35:49.281674 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:35:49.282575 (+ 901us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:35:49.282586 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:35:49.290465 (+ 7879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:35:49.290478 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:35:49.290608 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:35:49.291100 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:35:49.291120 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:35:49.293755 (+ 2635us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:35:49.295886 (+ 2131us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:35:49.295889 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:35:49.295891 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:35:49.296173 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:35:49.297711 (+ 1538us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:35:49.297715 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:35:49.297717 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:35:49.297855 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:35:49.298659 (+ 804us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:35:49.298662 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:35:49.298792 (+ 130us) client_negotiation.cc:769] Sending connection context
0807 18:35:49.298907 (+ 115us) client_negotiation.cc:240] Negotiation successful
0807 18:35:49.299073 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6551}
I20260807 18:35:49.299801 25850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:35:49.273180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51982 (local address 127.11.14.125:36793)
0807 18:35:49.273304 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0807 18:35:49.273308 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:35:49.279829 (+ 6521us) server_negotiation.cc:407] Connection header received
0807 18:35:49.279876 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:35:49.279879 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:35:49.279930 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:35:49.280017 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:35:49.280682 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:35:49.281487 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:35:49.289850 (+ 8363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:35:49.290189 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:35:49.290293 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:35:49.293912 (+ 3619us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:35:49.293916 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:35:49.293918 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:35:49.293941 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:35:49.295758 (+ 1817us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:35:49.297489 (+ 1731us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:35:49.297493 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:35:49.297495 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:35:49.297585 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:35:49.298216 (+ 631us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:35:49.298222 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:35:49.298226 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:35:49.298516 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:35:49.298607 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0807 18:35:49.299497 (+ 890us) server_negotiation.cc:299] Negotiation successful
0807 18:35:49.299647 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260807 18:35:49.299753 25951 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4e83dc1132fe48908f77dcc7e200f946 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5532e9367168440c949f3e3a203cef06" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 40395 } } peers { permanent_uuid: "4e83dc1132fe48908f77dcc7e200f946" member_type: VOTER last_known_addr { host: "127.11.14.125" port: 36793 } } peers { permanent_uuid: "9516f04f2b484fcba774e527313b028d" member_type: VOTER last_known_addr { host: "127.11.14.124" port: 37595 } } }
I20260807 18:35:49.299885 25951 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4e83dc1132fe48908f77dcc7e200f946 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 18:35:49.300431 25927 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.14.124" port: 37595 } has no permanent_uuid. Determining permanent_uuid...
I20260807 18:35:49.298835 25770 master_runner.cc:186] Error getting master registration for 127.11.14.124:37595: OK, instance_id { permanent_uuid: "9516f04f2b484fcba774e527313b028d" instance_seqno: 1786127749238168 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 18:35:49.302275 25770 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
Aug 07 18:35:49 dist-test-slave-6k22 krb5kdc[25671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127749, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.124@KRBTEST.COM for kudu/127.11.14.124@KRBTEST.COM
W20260807 18:35:49.307507 25965 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4e83dc1132fe48908f77dcc7e200f946: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 18:35:49.307614 25965 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 18:35:49.311050 25932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:35:49.300891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.124:37595 (local address 127.0.0.1:49674)
0807 18:35:49.301517 (+ 626us) negotiation.cc:107] Waiting for socket to connect
0807 18:35:49.301530 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 18:35:49.301639 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:35:49.301944 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:35:49.301946 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:35:49.302118 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:35:49.302218 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:35:49.302225 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:35:49.303315 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:35:49.303317 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:35:49.304174 (+ 857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:35:49.304187 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:35:49.304835 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:35:49.304838 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:35:49.304910 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:35:49.305808 (+ 898us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:35:49.305821 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:35:49.307533 (+ 1712us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:35:49.309560 (+ 2027us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:35:49.309563 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:35:49.309565 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:35:49.309810 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:35:49.310104 (+ 294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:35:49.310107 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:35:49.310109 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:35:49.310185 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:35:49.310572 (+ 387us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:35:49.310574 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:35:49.310639 (+ 65us) client_negotiation.cc:769] Sending connection context
0807 18:35:49.310733 (+ 94us) client_negotiation.cc:240] Negotiation successful
0807 18:35:49.310870 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":560,"thread_start_us":97,"threads_started":1}
I20260807 18:35:49.311682 25930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:35:49.301220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49674 (local address 127.11.14.124:37595)
0807 18:35:49.301330 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0807 18:35:49.301350 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0807 18:35:49.301581 (+ 231us) server_negotiation.cc:407] Connection header received
0807 18:35:49.301733 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:35:49.301736 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:35:49.301774 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:35:49.301819 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:35:49.302328 (+ 509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:35:49.303201 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:35:49.304287 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:35:49.304640 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:35:49.304712 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:35:49.307677 (+ 2965us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:35:49.307681 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:35:49.307684 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:35:49.307713 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:35:49.309459 (+ 1746us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:35:49.309918 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:35:49.309921 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:35:49.309923 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:35:49.310007 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:35:49.310268 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:35:49.310271 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:35:49.310272 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:35:49.310457 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:35:49.310521 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0807 18:35:49.311375 (+ 854us) server_negotiation.cc:299] Negotiation successful
0807 18:35:49.311486 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260807 18:35:49.315220 25934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:35:49.298444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43848 (local address 127.11.14.126:40395)
0807 18:35:49.298571 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0807 18:35:49.298578 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:35:49.300496 (+ 1918us) server_negotiation.cc:407] Connection header received
0807 18:35:49.302073 (+ 1577us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:35:49.302077 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:35:49.302132 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:35:49.302185 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:35:49.303206 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:35:49.304047 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:35:49.305211 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:35:49.305573 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:35:49.305676 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:35:49.311182 (+ 5506us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:35:49.311186 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:35:49.311189 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:35:49.311219 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:35:49.312943 (+ 1724us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:35:49.313455 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:35:49.313459 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:35:49.313461 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:35:49.313552 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:35:49.313845 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:35:49.313848 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:35:49.313850 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:35:49.314066 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:35:49.314164 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0807 18:35:49.314938 (+ 774us) server_negotiation.cc:299] Negotiation successful
0807 18:35:49.315067 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260807 18:35:49.315519 25927 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9516f04f2b484fcba774e527313b028d: Bootstrap starting.
I20260807 18:35:49.318722 25936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:35:49.298652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52000 (local address 127.11.14.125:36793)
0807 18:35:49.298924 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0807 18:35:49.298928 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:35:49.300370 (+ 1442us) server_negotiation.cc:407] Connection header received
0807 18:35:49.300854 (+ 484us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:35:49.300857 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:35:49.300903 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:35:49.300956 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:35:49.305261 (+ 4305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:35:49.306009 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:35:49.308890 (+ 2881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:35:49.309297 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:35:49.309391 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:35:49.314526 (+ 5135us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:35:49.314531 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:35:49.314533 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:35:49.314562 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:35:49.316627 (+ 2065us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:35:49.317245 (+ 618us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:35:49.317249 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:35:49.317250 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:35:49.317359 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:35:49.317686 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:35:49.317690 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:35:49.317691 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:35:49.317913 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:35:49.318037 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0807 18:35:49.318437 (+ 400us) server_negotiation.cc:299] Negotiation successful
0807 18:35:49.318565 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":163}
I20260807 18:35:49.318869 25927 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9516f04f2b484fcba774e527313b028d: Neither blocks nor log segments found. Creating new log.
I20260807 18:35:49.319980 25927 log.cc:824] T 00000000000000000000000000000000 P 9516f04f2b484fcba774e527313b028d: Log is configured to *not* fsync() on all Append() calls
I20260807 18:35:49.321605 25937 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5532e9367168440c949f3e3a203cef06 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:35:49.321779 25937 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5532e9367168440c949f3e3a203cef06 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5532e9367168440c949f3e3a203cef06" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 40395 } } peers { permanent_uuid: "4e83dc1132fe48908f77dcc7e200f946" member_type: VOTER last_known_addr { host: "127.11.14.125" port: 36793 } } peers { permanent_uuid: "9516f04f2b484fcba774e527313b028d" member_type: VOTER last_known_addr { host: "127.11.14.124" port: 37595 } }
I20260807 18:35:49.321982 25933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:35:49.298128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49666 (local address 127.11.14.124:37595)
0807 18:35:49.299706 (+ 1578us) server_negotiation.cc:206] Beginning negotiation
0807 18:35:49.299711 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:35:49.307470 (+ 7759us) server_negotiation.cc:407] Connection header received
0807 18:35:49.307744 (+ 274us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:35:49.307750 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:35:49.307797 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:35:49.307870 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:35:49.309925 (+ 2055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:35:49.310665 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:35:49.312542 (+ 1877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:35:49.312905 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:35:49.312986 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:35:49.314598 (+ 1612us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:35:49.314602 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:35:49.314604 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:35:49.314628 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:35:49.316994 (+ 2366us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:35:49.318727 (+ 1733us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:35:49.318733 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:35:49.318738 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:35:49.318844 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:35:49.320665 (+ 1821us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:35:49.320670 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:35:49.320672 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:35:49.320910 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:35:49.321066 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0807 18:35:49.321716 (+ 650us) server_negotiation.cc:299] Negotiation successful
0807 18:35:49.321845 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1500}
I20260807 18:35:49.322978 25937 leader_election.cc:288] T 00000000000000000000000000000000 P 5532e9367168440c949f3e3a203cef06 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4e83dc1132fe48908f77dcc7e200f946 (127.11.14.125:36793), 9516f04f2b484fcba774e527313b028d (127.11.14.124:37595)
I20260807 18:35:49.324054 25815 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5532e9367168440c949f3e3a203cef06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e83dc1132fe48908f77dcc7e200f946" is_pre_election: true
I20260807 18:35:49.324393 25815 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 4e83dc1132fe48908f77dcc7e200f946 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5532e9367168440c949f3e3a203cef06 in term 0.
I20260807 18:35:49.324913 25707 leader_election.cc:302] T 00000000000000000000000000000000 P 5532e9367168440c949f3e3a203cef06 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e83dc1132fe48908f77dcc7e200f946, 5532e9367168440c949f3e3a203cef06; no voters:
I20260807 18:35:49.325199 25937 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5532e9367168440c949f3e3a203cef06 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:35:49.325314 25937 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5532e9367168440c949f3e3a203cef06 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:35:49.325371 25937 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5532e9367168440c949f3e3a203cef06 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:35:49.326738 25937 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5532e9367168440c949f3e3a203cef06 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5532e9367168440c949f3e3a203cef06" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 40395 } } peers { permanent_uuid: "4e83dc1132fe48908f77dcc7e200f946" member_type: VOTER last_known_addr { host: "127.11.14.125" port: 36793 } } peers { permanent_uuid: "9516f04f2b484fcba774e527313b028d" member_type: VOTER last_known_addr { host: "127.11.14.124" port: 37595 } }
I20260807 18:35:49.327103 25937 leader_election.cc:288] T 00000000000000000000000000000000 P 5532e9367168440c949f3e3a203cef06 [CANDIDATE]: Term 1 election: Requested vote from peers 4e83dc1132fe48908f77dcc7e200f946 (127.11.14.125:36793), 9516f04f2b484fcba774e527313b028d (127.11.14.124:37595)
I20260807 18:35:49.330338 25815 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5532e9367168440c949f3e3a203cef06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e83dc1132fe48908f77dcc7e200f946"
I20260807 18:35:49.330477 25815 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4e83dc1132fe48908f77dcc7e200f946 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:35:49.331929 25815 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 4e83dc1132fe48908f77dcc7e200f946 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5532e9367168440c949f3e3a203cef06 in term 1.
I20260807 18:35:49.332237 25707 leader_election.cc:302] T 00000000000000000000000000000000 P 5532e9367168440c949f3e3a203cef06 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e83dc1132fe48908f77dcc7e200f946, 5532e9367168440c949f3e3a203cef06; no voters:
I20260807 18:35:49.332436 25937 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5532e9367168440c949f3e3a203cef06 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:35:49.332746 25937 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5532e9367168440c949f3e3a203cef06 [term 1 LEADER]: Becoming Leader. State: Replica: 5532e9367168440c949f3e3a203cef06, State: Running, Role: LEADER
I20260807 18:35:49.333153 25937 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5532e9367168440c949f3e3a203cef06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5532e9367168440c949f3e3a203cef06" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 40395 } } peers { permanent_uuid: "4e83dc1132fe48908f77dcc7e200f946" member_type: VOTER last_known_addr { host: "127.11.14.125" port: 36793 } } peers { permanent_uuid: "9516f04f2b484fcba774e527313b028d" member_type: VOTER last_known_addr { host: "127.11.14.124" port: 37595 } }
I20260807 18:35:49.337598 25927 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9516f04f2b484fcba774e527313b028d: No bootstrap required, opened a new log
I20260807 18:35:49.337899 25977 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5532e9367168440c949f3e3a203cef06 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5532e9367168440c949f3e3a203cef06. Latest consensus state: current_term: 1 leader_uuid: "5532e9367168440c949f3e3a203cef06" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5532e9367168440c949f3e3a203cef06" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 40395 } } peers { permanent_uuid: "4e83dc1132fe48908f77dcc7e200f946" member_type: VOTER last_known_addr { host: "127.11.14.125" port: 36793 } } peers { permanent_uuid: "9516f04f2b484fcba774e527313b028d" member_type: VOTER last_known_addr { host: "127.11.14.124" port: 37595 } } }
I20260807 18:35:49.338302 25977 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5532e9367168440c949f3e3a203cef06 [sys.catalog]: This master's current role is: LEADER
I20260807 18:35:49.339179 25979 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:35:49.341540 25979 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:35:49.342945 25927 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9516f04f2b484fcba774e527313b028d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5532e9367168440c949f3e3a203cef06" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 40395 } } peers { permanent_uuid: "4e83dc1132fe48908f77dcc7e200f946" member_type: VOTER last_known_addr { host: "127.11.14.125" port: 36793 } } peers { permanent_uuid: "9516f04f2b484fcba774e527313b028d" member_type: VOTER last_known_addr { host: "127.11.14.124" port: 37595 } }
I20260807 18:35:49.343225 25927 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9516f04f2b484fcba774e527313b028d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:35:49.345988 25927 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9516f04f2b484fcba774e527313b028d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9516f04f2b484fcba774e527313b028d, State: Initialized, Role: FOLLOWER
I20260807 18:35:49.346552 25927 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9516f04f2b484fcba774e527313b028d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5532e9367168440c949f3e3a203cef06" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 40395 } } peers { permanent_uuid: "4e83dc1132fe48908f77dcc7e200f946" member_type: VOTER last_known_addr { host: "127.11.14.125" port: 36793 } } peers { permanent_uuid: "9516f04f2b484fcba774e527313b028d" member_type: VOTER last_known_addr { host: "127.11.14.124" port: 37595 } }
I20260807 18:35:49.347625 25980 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9516f04f2b484fcba774e527313b028d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5532e9367168440c949f3e3a203cef06" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 40395 } } peers { permanent_uuid: "4e83dc1132fe48908f77dcc7e200f946" member_type: VOTER last_known_addr { host: "127.11.14.125" port: 36793 } } peers { permanent_uuid: "9516f04f2b484fcba774e527313b028d" member_type: VOTER last_known_addr { host: "127.11.14.124" port: 37595 } } }
I20260807 18:35:49.350196 25980 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9516f04f2b484fcba774e527313b028d [sys.catalog]: This master's current role is: FOLLOWER
I20260807 18:35:49.351400 25897 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9516f04f2b484fcba774e527313b028d [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:35:49.351972 25815 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 4e83dc1132fe48908f77dcc7e200f946 [term 1 FOLLOWER]: Refusing update from remote peer 5532e9367168440c949f3e3a203cef06: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:35:49.348153 25927 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9516f04f2b484fcba774e527313b028d [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:35:49.348194 25898 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5532e9367168440c949f3e3a203cef06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9516f04f2b484fcba774e527313b028d"
I20260807 18:35:49.348318 25899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5532e9367168440c949f3e3a203cef06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9516f04f2b484fcba774e527313b028d" is_pre_election: true
I20260807 18:35:49.352849 25897 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 9516f04f2b484fcba774e527313b028d [term 1 FOLLOWER]: Refusing update from remote peer 5532e9367168440c949f3e3a203cef06: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:35:49.353418 25937 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 5532e9367168440c949f3e3a203cef06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e83dc1132fe48908f77dcc7e200f946" member_type: VOTER last_known_addr { host: "127.11.14.125" port: 36793 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:35:49.353899 25937 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 5532e9367168440c949f3e3a203cef06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9516f04f2b484fcba774e527313b028d" member_type: VOTER last_known_addr { host: "127.11.14.124" port: 37595 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:35:49.360644 25951 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4e83dc1132fe48908f77dcc7e200f946 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5532e9367168440c949f3e3a203cef06. Latest consensus state: current_term: 1 leader_uuid: "5532e9367168440c949f3e3a203cef06" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5532e9367168440c949f3e3a203cef06" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 40395 } } peers { permanent_uuid: "4e83dc1132fe48908f77dcc7e200f946" member_type: VOTER last_known_addr { host: "127.11.14.125" port: 36793 } } peers { permanent_uuid: "9516f04f2b484fcba774e527313b028d" member_type: VOTER last_known_addr { host: "127.11.14.124" port: 37595 } } }
I20260807 18:35:49.360775 25951 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4e83dc1132fe48908f77dcc7e200f946 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 18:35:49.361513 25977 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5532e9367168440c949f3e3a203cef06 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5532e9367168440c949f3e3a203cef06" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5532e9367168440c949f3e3a203cef06" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 40395 } } peers { permanent_uuid: "4e83dc1132fe48908f77dcc7e200f946" member_type: VOTER last_known_addr { host: "127.11.14.125" port: 36793 } } peers { permanent_uuid: "9516f04f2b484fcba774e527313b028d" member_type: VOTER last_known_addr { host: "127.11.14.124" port: 37595 } } }
I20260807 18:35:49.361622 25977 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5532e9367168440c949f3e3a203cef06 [sys.catalog]: This master's current role is: LEADER
I20260807 18:35:49.363098 25979 catalog_manager.cc:1347] Generated new cluster ID: 5309e33959204d30a383e6ee20e1bc11
I20260807 18:35:49.363163 25979 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:35:49.365720 25995 mvcc.cc:205] Tried to move back new op lower bound from 7315979261336301568 to 7315979261272125440. Current Snapshot: MvccSnapshot[applied={T|T < 7315979261336301568}]
I20260807 18:35:49.365877 25951 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4e83dc1132fe48908f77dcc7e200f946 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5532e9367168440c949f3e3a203cef06" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5532e9367168440c949f3e3a203cef06" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 40395 } } peers { permanent_uuid: "4e83dc1132fe48908f77dcc7e200f946" member_type: VOTER last_known_addr { host: "127.11.14.125" port: 36793 } } peers { permanent_uuid: "9516f04f2b484fcba774e527313b028d" member_type: VOTER last_known_addr { host: "127.11.14.124" port: 37595 } } }
I20260807 18:35:49.365960 25951 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4e83dc1132fe48908f77dcc7e200f946 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 18:35:49.370739 25937 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5532e9367168440c949f3e3a203cef06 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5532e9367168440c949f3e3a203cef06" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5532e9367168440c949f3e3a203cef06" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 40395 } } peers { permanent_uuid: "4e83dc1132fe48908f77dcc7e200f946" member_type: VOTER last_known_addr { host: "127.11.14.125" port: 36793 } } peers { permanent_uuid: "9516f04f2b484fcba774e527313b028d" member_type: VOTER last_known_addr { host: "127.11.14.124" port: 37595 } } }
I20260807 18:35:49.370886 25937 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5532e9367168440c949f3e3a203cef06 [sys.catalog]: This master's current role is: LEADER
I20260807 18:35:49.360960 25980 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9516f04f2b484fcba774e527313b028d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5532e9367168440c949f3e3a203cef06. Latest consensus state: current_term: 1 leader_uuid: "5532e9367168440c949f3e3a203cef06" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5532e9367168440c949f3e3a203cef06" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 40395 } } peers { permanent_uuid: "4e83dc1132fe48908f77dcc7e200f946" member_type: VOTER last_known_addr { host: "127.11.14.125" port: 36793 } } peers { permanent_uuid: "9516f04f2b484fcba774e527313b028d" member_type: VOTER last_known_addr { host: "127.11.14.124" port: 37595 } } }
W20260807 18:35:49.360983 25992 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9516f04f2b484fcba774e527313b028d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 18:35:49.371656 25992 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 18:35:49.371795 25980 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9516f04f2b484fcba774e527313b028d [sys.catalog]: This master's current role is: FOLLOWER
I20260807 18:35:49.372676 25980 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9516f04f2b484fcba774e527313b028d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5532e9367168440c949f3e3a203cef06" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5532e9367168440c949f3e3a203cef06" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 40395 } } peers { permanent_uuid: "4e83dc1132fe48908f77dcc7e200f946" member_type: VOTER last_known_addr { host: "127.11.14.125" port: 36793 } } peers { permanent_uuid: "9516f04f2b484fcba774e527313b028d" member_type: VOTER last_known_addr { host: "127.11.14.124" port: 37595 } } }
I20260807 18:35:49.372859 25980 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9516f04f2b484fcba774e527313b028d [sys.catalog]: This master's current role is: FOLLOWER
I20260807 18:35:49.373492 25981 mvcc.cc:205] Tried to move back new op lower bound from 7315979261336301568 to 7315979261272125440. Current Snapshot: MvccSnapshot[applied={T|T < 7315979261336301568 or (T in {7315979261336301568})}]
I20260807 18:35:49.375672 25993 mvcc.cc:205] Tried to move back new op lower bound from 7315979261336301568 to 7315979261272125440. Current Snapshot: MvccSnapshot[applied={T|T < 7315979261336301568}]
I20260807 18:35:49.381882 25979 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:35:49.383652 25979 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:35:49.394346 25979 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5532e9367168440c949f3e3a203cef06: Generated new TSK 0
I20260807 18:35:49.395289 25979 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:35:50.309343 25965 catalog_manager.cc:1259] Loaded cluster ID: 5309e33959204d30a383e6ee20e1bc11
I20260807 18:35:50.309442 25965 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 4e83dc1132fe48908f77dcc7e200f946: loading cluster ID for follower catalog manager: success
I20260807 18:35:50.311189 25965 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 4e83dc1132fe48908f77dcc7e200f946: acquiring CA information for follower catalog manager: success
I20260807 18:35:50.312600 25965 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 4e83dc1132fe48908f77dcc7e200f946: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 18:35:50.373091 25992 catalog_manager.cc:1259] Loaded cluster ID: 5309e33959204d30a383e6ee20e1bc11
I20260807 18:35:50.373211 25992 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 9516f04f2b484fcba774e527313b028d: loading cluster ID for follower catalog manager: success
I20260807 18:35:50.374907 25992 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 9516f04f2b484fcba774e527313b028d: acquiring CA information for follower catalog manager: success
I20260807 18:35:50.376088 25992 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 9516f04f2b484fcba774e527313b028d: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 18:35:50.468956 11321 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkz56XN/build/debug/bin/kudu with pid 25686
I20260807 18:35:50.480436 11321 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkz56XN/build/debug/bin/kudu with pid 25770
I20260807 18:35:50.490087 11321 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkz56XN/build/debug/bin/kudu with pid 25854
2026-08-07T18:35:50Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3291 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786127579900523-11321-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:35:50 dist-test-slave-6k22 krb5kdc[26015](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:35:50 dist-test-slave-6k22 krb5kdc[26015](info): set up 2 sockets
Aug 07 18:35:50 dist-test-slave-6k22 krb5kdc[26015](info): commencing operation
krb5kdc: starting...
W20260807 18:35:51.608930 11321 mini_kdc.cc:121] Time spent starting KDC: real 1.093s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786127579900523-11321-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786127579900523-11321-0/krb5kdc/test-user.keytab.
Aug 07 18:35:51 dist-test-slave-6k22 krb5kdc[26015](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127751, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:35:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:35:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.14.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.14.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.14.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.126.keytab.
Entry for principal kudu/127.11.14.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.126.keytab.
I20260807 18:35:51.747697 11321 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkz56XN/build/debug/bin/kudu
/tmp/dist-test-taskkz56XN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786127579900523-11321-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786127579900523-11321-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786127579900523-11321-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786127579900523-11321-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.14.126:37229
--webserver_interface=127.11.14.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.126.keytab
--principal=kudu/127.11.14.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.14.84:37817
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.14.126:37229
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786127579900523-11321-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786127579900523-11321-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786127579900523-11321-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:35:51.869673 26028 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:35:51.870002 26028 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:35:51.870085 26028 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:35:51.874109 26028 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:35:51.874209 26028 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:35:51.874248 26028 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:35:51.874282 26028 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:35:51.874310 26028 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:35:51.879647 26028 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.14.84:37817
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786127579900523-11321-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786127579900523-11321-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.14.126:37229
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.126.keytab
--principal=kudu/127.11.14.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.14.126:37229
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786127579900523-11321-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.14.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26028
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786127579900523-11321-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:35:51.880801 26028 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:35:51.881888 26028 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:35:51.890937 26035 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:35:51.890975 26034 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:35:51.890951 26037 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:35:51.891059 26028 server_base.cc:1034] running on GCE node
I20260807 18:35:51.891861 26028 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:35:51.893024 26028 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:35:51.894214 26028 hybrid_clock.cc:648] HybridClock initialized: now 1786127751894199 us; error 26 us; skew 500 ppm
Aug 07 18:35:51 dist-test-slave-6k22 krb5kdc[26015](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127751, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:35:51.897102 26028 init.cc:377] Logged in from keytab as kudu/127.11.14.126@KRBTEST.COM (short username kudu)
I20260807 18:35:51.898164 26028 webserver.cc:460] Webserver started at http://127.11.14.126:36699/ using document root <none> and password file <none>
I20260807 18:35:51.898744 26028 fs_manager.cc:359] Metadata directory not provided
I20260807 18:35:51.898828 26028 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:35:51.899024 26028 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:35:51.901093 26028 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786127579900523-11321-0/minicluster-data/master-0/data/instance:
uuid: "14b4ca0d85d64986b3a9943e604800f0"
format_stamp: "Formatted at 2026-08-07 18:35:51 on dist-test-slave-6k22"
server_key: "6341cca450e21ee2e6590cdb614e7f47"
server_key_iv: "bda69ed6a960f7d3d2fce142af7ec1e6"
server_key_version: "encryptionkey@0"
I20260807 18:35:51.901625 26028 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786127579900523-11321-0/minicluster-data/master-0/wal/instance:
uuid: "14b4ca0d85d64986b3a9943e604800f0"
format_stamp: "Formatted at 2026-08-07 18:35:51 on dist-test-slave-6k22"
server_key: "6341cca450e21ee2e6590cdb614e7f47"
server_key_iv: "bda69ed6a960f7d3d2fce142af7ec1e6"
server_key_version: "encryptionkey@0"
I20260807 18:35:51.905514 26028 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 18:35:51.908082 26045 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:35:51.909448 26028 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:35:51.909588 26028 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786127579900523-11321-0/minicluster-data/master-0/data,/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786127579900523-11321-0/minicluster-data/master-0/wal
uuid: "14b4ca0d85d64986b3a9943e604800f0"
format_stamp: "Formatted at 2026-08-07 18:35:51 on dist-test-slave-6k22"
server_key: "6341cca450e21ee2e6590cdb614e7f47"
server_key_iv: "bda69ed6a960f7d3d2fce142af7ec1e6"
server_key_version: "encryptionkey@0"
I20260807 18:35:51.909705 26028 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786127579900523-11321-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786127579900523-11321-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786127579900523-11321-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:35:51.928365 26028 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:35:51.932135 26028 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:35:51.932376 26028 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:35:51.940431 26028 rpc_server.cc:307] RPC server started. Bound to: 127.11.14.126:37229
I20260807 18:35:51.940459 26100 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.14.126:37229 every 8 connection(s)
I20260807 18:35:51.941511 26028 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786127579900523-11321-0/minicluster-data/master-0/data/info.pb
I20260807 18:35:51.944661 26101 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:35:51.945979 11321 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkz56XN/build/debug/bin/kudu as pid 26028
I20260807 18:35:51.946081 11321 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786127579900523-11321-0/minicluster-data/master-0/wal/instance
I20260807 18:35:51.946425 11321 external_mini_cluster.cc:1428] Setting key 496be68e7ac834c8cc7326f14b64556d
I20260807 18:35:51.951103 26101 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 14b4ca0d85d64986b3a9943e604800f0: Bootstrap starting.
I20260807 18:35:51.953570 26101 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 14b4ca0d85d64986b3a9943e604800f0: Neither blocks nor log segments found. Creating new log.
I20260807 18:35:51.954356 26101 log.cc:824] T 00000000000000000000000000000000 P 14b4ca0d85d64986b3a9943e604800f0: Log is configured to *not* fsync() on all Append() calls
Aug 07 18:35:51 dist-test-slave-6k22 krb5kdc[26015](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127751, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.14.126@KRBTEST.COM
I20260807 18:35:51.956800 26101 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 14b4ca0d85d64986b3a9943e604800f0: No bootstrap required, opened a new log
I20260807 18:35:51.961637 26106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:35:51.947964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54954 (local address 127.11.14.126:37229)
0807 18:35:51.948426 (+ 462us) server_negotiation.cc:206] Beginning negotiation
0807 18:35:51.948444 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0807 18:35:51.948482 (+ 38us) server_negotiation.cc:407] Connection header received
0807 18:35:51.949137 (+ 655us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:35:51.949154 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:35:51.949507 (+ 353us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:35:51.949869 (+ 362us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:35:51.950729 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:35:51.951826 (+ 1097us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:35:51.952708 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:35:51.953212 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:35:51.953399 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:35:51.956145 (+ 2746us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:35:51.956152 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:35:51.956169 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:35:51.956201 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:35:51.958221 (+ 2020us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:35:51.958673 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:35:51.958681 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:35:51.958689 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:35:51.958843 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:35:51.959113 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:35:51.959117 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:35:51.959120 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:35:51.959768 (+ 648us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:35:51.959976 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0807 18:35:51.960280 (+ 304us) server_negotiation.cc:299] Negotiation successful
0807 18:35:51.960604 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":303,"thread_start_us":138,"threads_started":1}
I20260807 18:35:51.961683 26101 raft_consensus.cc:348] T 00000000000000000000000000000000 P 14b4ca0d85d64986b3a9943e604800f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14b4ca0d85d64986b3a9943e604800f0" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 37229 } }
I20260807 18:35:51.962008 26101 raft_consensus.cc:374] T 00000000000000000000000000000000 P 14b4ca0d85d64986b3a9943e604800f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:35:51.962090 26101 raft_consensus.cc:729] T 00000000000000000000000000000000 P 14b4ca0d85d64986b3a9943e604800f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 14b4ca0d85d64986b3a9943e604800f0, State: Initialized, Role: FOLLOWER
I20260807 18:35:51.962581 26101 consensus_queue.cc:260] T 00000000000000000000000000000000 P 14b4ca0d85d64986b3a9943e604800f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14b4ca0d85d64986b3a9943e604800f0" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 37229 } }
I20260807 18:35:51.962754 26101 raft_consensus.cc:388] T 00000000000000000000000000000000 P 14b4ca0d85d64986b3a9943e604800f0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:35:51.962828 26101 raft_consensus.cc:482] T 00000000000000000000000000000000 P 14b4ca0d85d64986b3a9943e604800f0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:35:51.962924 26101 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 14b4ca0d85d64986b3a9943e604800f0 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:35:51.964107 26101 raft_consensus.cc:504] T 00000000000000000000000000000000 P 14b4ca0d85d64986b3a9943e604800f0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14b4ca0d85d64986b3a9943e604800f0" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 37229 } }
I20260807 18:35:51.964612 26101 leader_election.cc:302] T 00000000000000000000000000000000 P 14b4ca0d85d64986b3a9943e604800f0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 14b4ca0d85d64986b3a9943e604800f0; no voters:
I20260807 18:35:51.965059 11321 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 18:35:51.965096 26101 leader_election.cc:288] T 00000000000000000000000000000000 P 14b4ca0d85d64986b3a9943e604800f0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:35:51.965212 26108 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 14b4ca0d85d64986b3a9943e604800f0 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:35:51.965464 26108 raft_consensus.cc:686] T 00000000000000000000000000000000 P 14b4ca0d85d64986b3a9943e604800f0 [term 1 LEADER]: Becoming Leader. State: Replica: 14b4ca0d85d64986b3a9943e604800f0, State: Running, Role: LEADER
I20260807 18:35:51.965837 26108 consensus_queue.cc:237] T 00000000000000000000000000000000 P 14b4ca0d85d64986b3a9943e604800f0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14b4ca0d85d64986b3a9943e604800f0" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 37229 } }
I20260807 18:35:51.966511 26101 sys_catalog.cc:564] T 00000000000000000000000000000000 P 14b4ca0d85d64986b3a9943e604800f0 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:35:51.967914 26109 sys_catalog.cc:455] T 00000000000000000000000000000000 P 14b4ca0d85d64986b3a9943e604800f0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "14b4ca0d85d64986b3a9943e604800f0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14b4ca0d85d64986b3a9943e604800f0" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 37229 } } }
I20260807 18:35:51.968055 26109 sys_catalog.cc:458] T 00000000000000000000000000000000 P 14b4ca0d85d64986b3a9943e604800f0 [sys.catalog]: This master's current role is: LEADER
I20260807 18:35:51.968394 26110 sys_catalog.cc:455] T 00000000000000000000000000000000 P 14b4ca0d85d64986b3a9943e604800f0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 14b4ca0d85d64986b3a9943e604800f0. Latest consensus state: current_term: 1 leader_uuid: "14b4ca0d85d64986b3a9943e604800f0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14b4ca0d85d64986b3a9943e604800f0" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 37229 } } }
I20260807 18:35:51.968523 26110 sys_catalog.cc:458] T 00000000000000000000000000000000 P 14b4ca0d85d64986b3a9943e604800f0 [sys.catalog]: This master's current role is: LEADER
I20260807 18:35:51.968466 26119 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:35:51.971414 26119 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:35:51.977630 26119 catalog_manager.cc:1347] Generated new cluster ID: 1b73c9c963c04db3a158cba16315930c
I20260807 18:35:51.977720 26119 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:35:51.992715 26119 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:35:51.994091 26119 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:35:52.007027 26119 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 14b4ca0d85d64986b3a9943e604800f0: Generated new TSK 0
I20260807 18:35:52.008044 26119 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:35:52.021661 11321 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkz56XN/build/debug/bin/kudu with pid 26028
2026-08-07T18:35:52Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1530 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:35:52 dist-test-slave-6k22 krb5kdc[26133](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:35:52 dist-test-slave-6k22 krb5kdc[26133](info): set up 2 sockets
Aug 07 18:35:52 dist-test-slave-6k22 krb5kdc[26133](info): commencing operation
krb5kdc: starting...
W20260807 18:35:53.126416 11321 mini_kdc.cc:121] Time spent starting KDC: real 1.080s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/krb5kdc/test-user.keytab.
Aug 07 18:35:53 dist-test-slave-6k22 krb5kdc[26133](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127753, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:35:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:35:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.14.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.14.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.14.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.126.keytab.
Entry for principal kudu/127.11.14.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.126.keytab.
I20260807 18:35:53.263602 11321 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkz56XN/build/debug/bin/kudu
/tmp/dist-test-taskkz56XN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.14.126:41615
--webserver_interface=127.11.14.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.126.keytab
--principal=kudu/127.11.14.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.14.84:39981
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.14.126:41615
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:35:53.384636 26150 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:35:53.384989 26150 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:35:53.385074 26150 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:35:53.389231 26150 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:35:53.389353 26150 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:35:53.389391 26150 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:35:53.389427 26150 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:35:53.389456 26150 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:35:53.395066 26150 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.14.84:39981
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.14.126:41615
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.126.keytab
--principal=kudu/127.11.14.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.14.126:41615
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.14.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26150
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:35:53.396418 26150 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:35:53.397604 26150 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:35:53.407155 26157 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:35:53.407337 26150 server_base.cc:1034] running on GCE node
W20260807 18:35:53.407150 26159 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:35:53.407584 26156 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:35:53.408243 26150 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:35:53.409529 26150 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:35:53.410712 26150 hybrid_clock.cc:648] HybridClock initialized: now 1786127753410682 us; error 43 us; skew 500 ppm
Aug 07 18:35:53 dist-test-slave-6k22 krb5kdc[26133](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127753, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:35:53.419332 26150 init.cc:377] Logged in from keytab as kudu/127.11.14.126@KRBTEST.COM (short username kudu)
I20260807 18:35:53.420534 26150 webserver.cc:460] Webserver started at http://127.11.14.126:44139/ using document root <none> and password file <none>
I20260807 18:35:53.421144 26150 fs_manager.cc:359] Metadata directory not provided
I20260807 18:35:53.421208 26150 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:35:53.421394 26150 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:35:53.423544 26150 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/minicluster-data/master-0/data/instance:
uuid: "c582c50c1bb949ab9b83a517755fc867"
format_stamp: "Formatted at 2026-08-07 18:35:53 on dist-test-slave-6k22"
server_key: "a6c139fcb30d72205b96015e416b6a55"
server_key_iv: "2e9c60b703742dee832cf5a9347181a9"
server_key_version: "encryptionkey@0"
I20260807 18:35:53.424198 26150 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/minicluster-data/master-0/wal/instance:
uuid: "c582c50c1bb949ab9b83a517755fc867"
format_stamp: "Formatted at 2026-08-07 18:35:53 on dist-test-slave-6k22"
server_key: "a6c139fcb30d72205b96015e416b6a55"
server_key_iv: "2e9c60b703742dee832cf5a9347181a9"
server_key_version: "encryptionkey@0"
I20260807 18:35:53.428359 26150 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260807 18:35:53.431151 26166 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:35:53.432581 26150 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 18:35:53.432703 26150 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/minicluster-data/master-0/data,/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/minicluster-data/master-0/wal
uuid: "c582c50c1bb949ab9b83a517755fc867"
format_stamp: "Formatted at 2026-08-07 18:35:53 on dist-test-slave-6k22"
server_key: "a6c139fcb30d72205b96015e416b6a55"
server_key_iv: "2e9c60b703742dee832cf5a9347181a9"
server_key_version: "encryptionkey@0"
I20260807 18:35:53.432806 26150 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:35:53.447113 26150 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:35:53.450413 26150 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:35:53.450584 26150 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:35:53.458420 26150 rpc_server.cc:307] RPC server started. Bound to: 127.11.14.126:41615
I20260807 18:35:53.458738 26220 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.14.126:41615 every 8 connection(s)
I20260807 18:35:53.459535 26150 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/minicluster-data/master-0/data/info.pb
I20260807 18:35:53.461577 11321 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkz56XN/build/debug/bin/kudu as pid 26150
I20260807 18:35:53.461683 11321 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/minicluster-data/master-0/wal/instance
I20260807 18:35:53.461985 11321 external_mini_cluster.cc:1428] Setting key 8ceb13d69927580a71bc2b746b41407f
I20260807 18:35:53.462752 26221 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:35:53.469954 26221 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c582c50c1bb949ab9b83a517755fc867: Bootstrap starting.
Aug 07 18:35:53 dist-test-slave-6k22 krb5kdc[26133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127753, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.14.126@KRBTEST.COM
I20260807 18:35:53.472820 26221 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c582c50c1bb949ab9b83a517755fc867: Neither blocks nor log segments found. Creating new log.
I20260807 18:35:53.473716 26221 log.cc:824] T 00000000000000000000000000000000 P c582c50c1bb949ab9b83a517755fc867: Log is configured to *not* fsync() on all Append() calls
I20260807 18:35:53.475895 26221 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c582c50c1bb949ab9b83a517755fc867: No bootstrap required, opened a new log
I20260807 18:35:53.477046 26225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:35:53.463455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44788 (local address 127.11.14.126:41615)
0807 18:35:53.463958 (+ 503us) server_negotiation.cc:206] Beginning negotiation
0807 18:35:53.463982 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0807 18:35:53.464020 (+ 38us) server_negotiation.cc:407] Connection header received
0807 18:35:53.464664 (+ 644us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:35:53.464680 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:35:53.465028 (+ 348us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:35:53.465315 (+ 287us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:35:53.466334 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:35:53.467340 (+ 1006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:35:53.468575 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:35:53.469051 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:35:53.469171 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:35:53.472119 (+ 2948us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:35:53.472125 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:35:53.472140 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:35:53.472171 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:35:53.474073 (+ 1902us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:35:53.474552 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:35:53.474558 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:35:53.474564 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:35:53.474682 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:35:53.474982 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:35:53.474985 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:35:53.474987 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:35:53.475416 (+ 429us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:35:53.475661 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0807 18:35:53.475985 (+ 324us) server_negotiation.cc:299] Negotiation successful
0807 18:35:53.476306 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":354,"thread_start_us":171,"threads_started":1}
I20260807 18:35:53.479518 26221 raft_consensus.cc:348] T 00000000000000000000000000000000 P c582c50c1bb949ab9b83a517755fc867 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c582c50c1bb949ab9b83a517755fc867" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 41615 } }
I20260807 18:35:53.479848 26221 raft_consensus.cc:374] T 00000000000000000000000000000000 P c582c50c1bb949ab9b83a517755fc867 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:35:53.479959 26221 raft_consensus.cc:729] T 00000000000000000000000000000000 P c582c50c1bb949ab9b83a517755fc867 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c582c50c1bb949ab9b83a517755fc867, State: Initialized, Role: FOLLOWER
I20260807 18:35:53.480397 26221 consensus_queue.cc:260] T 00000000000000000000000000000000 P c582c50c1bb949ab9b83a517755fc867 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c582c50c1bb949ab9b83a517755fc867" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 41615 } }
I20260807 18:35:53.480525 26221 raft_consensus.cc:388] T 00000000000000000000000000000000 P c582c50c1bb949ab9b83a517755fc867 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:35:53.480572 26221 raft_consensus.cc:482] T 00000000000000000000000000000000 P c582c50c1bb949ab9b83a517755fc867 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:35:53.480645 26221 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c582c50c1bb949ab9b83a517755fc867 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:35:53.481637 26221 raft_consensus.cc:504] T 00000000000000000000000000000000 P c582c50c1bb949ab9b83a517755fc867 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c582c50c1bb949ab9b83a517755fc867" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 41615 } }
I20260807 18:35:53.481949 26221 leader_election.cc:302] T 00000000000000000000000000000000 P c582c50c1bb949ab9b83a517755fc867 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c582c50c1bb949ab9b83a517755fc867; no voters:
I20260807 18:35:53.482293 26221 leader_election.cc:288] T 00000000000000000000000000000000 P c582c50c1bb949ab9b83a517755fc867 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:35:53.482458 26227 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c582c50c1bb949ab9b83a517755fc867 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:35:53.482808 26227 raft_consensus.cc:686] T 00000000000000000000000000000000 P c582c50c1bb949ab9b83a517755fc867 [term 1 LEADER]: Becoming Leader. State: Replica: c582c50c1bb949ab9b83a517755fc867, State: Running, Role: LEADER
I20260807 18:35:53.483199 26227 consensus_queue.cc:237] T 00000000000000000000000000000000 P c582c50c1bb949ab9b83a517755fc867 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c582c50c1bb949ab9b83a517755fc867" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 41615 } }
I20260807 18:35:53.483750 26221 sys_catalog.cc:564] T 00000000000000000000000000000000 P c582c50c1bb949ab9b83a517755fc867 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:35:53.485049 26230 sys_catalog.cc:455] T 00000000000000000000000000000000 P c582c50c1bb949ab9b83a517755fc867 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c582c50c1bb949ab9b83a517755fc867" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c582c50c1bb949ab9b83a517755fc867" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 41615 } } }
I20260807 18:35:53.485193 26230 sys_catalog.cc:458] T 00000000000000000000000000000000 P c582c50c1bb949ab9b83a517755fc867 [sys.catalog]: This master's current role is: LEADER
I20260807 18:35:53.485303 26231 sys_catalog.cc:455] T 00000000000000000000000000000000 P c582c50c1bb949ab9b83a517755fc867 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c582c50c1bb949ab9b83a517755fc867. Latest consensus state: current_term: 1 leader_uuid: "c582c50c1bb949ab9b83a517755fc867" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c582c50c1bb949ab9b83a517755fc867" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 41615 } } }
I20260807 18:35:53.485430 26231 sys_catalog.cc:458] T 00000000000000000000000000000000 P c582c50c1bb949ab9b83a517755fc867 [sys.catalog]: This master's current role is: LEADER
I20260807 18:35:53.485615 26238 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:35:53.488807 26238 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:35:53.494992 26238 catalog_manager.cc:1347] Generated new cluster ID: 46cec966d5954dee8323a97f8fce0c46
I20260807 18:35:53.495091 26238 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:35:53.506997 26238 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:35:53.508118 26238 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:35:53.514776 26238 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c582c50c1bb949ab9b83a517755fc867: Generated new TSK 0
I20260807 18:35:53.515743 26238 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.14.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.14.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.14.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.65.keytab.
Entry for principal kudu/127.11.14.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.65.keytab.
I20260807 18:35:53.555095 11321 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkz56XN/build/debug/bin/kudu
/tmp/dist-test-taskkz56XN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.14.65:0
--local_ip_for_outbound_sockets=127.11.14.65
--webserver_interface=127.11.14.65
--webserver_port=0
--tserver_master_addrs=127.11.14.126:41615
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.65.keytab
--principal=kudu/127.11.14.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.14.84:39981
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:35:53.675011 26259 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:35:53.675297 26259 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:35:53.675381 26259 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:35:53.679416 26259 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:35:53.679486 26259 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:35:53.679638 26259 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.14.65
I20260807 18:35:53.685573 26259 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.14.84:39981
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.65.keytab
--principal=kudu/127.11.14.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.14.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.14.65
--webserver_port=0
--tserver_master_addrs=127.11.14.126:41615
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26259
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.14.65
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:35:53.686805 26259 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:35:53.687960 26259 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:35:53.698875 26265 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:35:53.698897 26264 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:35:53.698869 26267 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:35:53.698943 26259 server_base.cc:1034] running on GCE node
I20260807 18:35:53.699525 26259 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:35:53.700300 26259 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:35:53.701472 26259 hybrid_clock.cc:648] HybridClock initialized: now 1786127753701457 us; error 25 us; skew 500 ppm
Aug 07 18:35:53 dist-test-slave-6k22 krb5kdc[26133](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127753, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:35:53.704617 26259 init.cc:377] Logged in from keytab as kudu/127.11.14.65@KRBTEST.COM (short username kudu)
I20260807 18:35:53.705581 26259 webserver.cc:460] Webserver started at http://127.11.14.65:37565/ using document root <none> and password file <none>
I20260807 18:35:53.706187 26259 fs_manager.cc:359] Metadata directory not provided
I20260807 18:35:53.706252 26259 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:35:53.706481 26259 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:35:53.708644 26259 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/minicluster-data/ts-0/data/instance:
uuid: "e015739a99844c14829e0b989348e2f1"
format_stamp: "Formatted at 2026-08-07 18:35:53 on dist-test-slave-6k22"
server_key: "f11acd2898004152d4748cf435a289b1"
server_key_iv: "803ddf15583642fd88ed99d9769544b7"
server_key_version: "encryptionkey@0"
I20260807 18:35:53.709227 26259 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/minicluster-data/ts-0/wal/instance:
uuid: "e015739a99844c14829e0b989348e2f1"
format_stamp: "Formatted at 2026-08-07 18:35:53 on dist-test-slave-6k22"
server_key: "f11acd2898004152d4748cf435a289b1"
server_key_iv: "803ddf15583642fd88ed99d9769544b7"
server_key_version: "encryptionkey@0"
I20260807 18:35:53.713158 26259 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 18:35:53.715848 26274 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:35:53.717310 26259 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:35:53.717459 26259 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/minicluster-data/ts-0/data,/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/minicluster-data/ts-0/wal
uuid: "e015739a99844c14829e0b989348e2f1"
format_stamp: "Formatted at 2026-08-07 18:35:53 on dist-test-slave-6k22"
server_key: "f11acd2898004152d4748cf435a289b1"
server_key_iv: "803ddf15583642fd88ed99d9769544b7"
server_key_version: "encryptionkey@0"
I20260807 18:35:53.717578 26259 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:35:53.736922 26259 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:35:53.740510 26259 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:35:53.740736 26259 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:35:53.741389 26259 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:35:53.742265 26259 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:35:53.742348 26259 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:35:53.742420 26259 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:35:53.742460 26259 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:35:53.752259 26259 rpc_server.cc:307] RPC server started. Bound to: 127.11.14.65:38891
I20260807 18:35:53.752280 26398 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.14.65:38891 every 8 connection(s)
I20260807 18:35:53.753216 26259 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/minicluster-data/ts-0/data/info.pb
Aug 07 18:35:53 dist-test-slave-6k22 krb5kdc[26133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127753, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.65@KRBTEST.COM for kudu/127.11.14.126@KRBTEST.COM
I20260807 18:35:53.763157 11321 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkz56XN/build/debug/bin/kudu as pid 26259
I20260807 18:35:53.763309 11321 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/minicluster-data/ts-0/wal/instance
I20260807 18:35:53.763720 11321 external_mini_cluster.cc:1428] Setting key db30e702b22a6b78fe5ea6de1f88a39b
I20260807 18:35:53.767618 26225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:35:53.755185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.65:60963 (local address 127.11.14.126:41615)
0807 18:35:53.755330 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0807 18:35:53.755334 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:35:53.756258 (+ 924us) server_negotiation.cc:407] Connection header received
0807 18:35:53.757588 (+ 1330us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:35:53.757591 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:35:53.757638 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:35:53.757768 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:35:53.758960 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:35:53.759505 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:35:53.760719 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:35:53.760949 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:35:53.761033 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:35:53.763769 (+ 2736us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:35:53.763773 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:35:53.763775 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:35:53.763833 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:35:53.765432 (+ 1599us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:35:53.765992 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:35:53.765996 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:35:53.765998 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:35:53.766084 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:35:53.766446 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:35:53.766450 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:35:53.766452 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:35:53.766644 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:35:53.766754 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0807 18:35:53.767288 (+ 534us) server_negotiation.cc:299] Negotiation successful
0807 18:35:53.767427 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260807 18:35:53.768466 26401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:35:53.755635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:41615 (local address 127.11.14.65:60963)
0807 18:35:53.756091 (+ 456us) negotiation.cc:107] Waiting for socket to connect
0807 18:35:53.756135 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 18:35:53.757377 (+ 1242us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:35:53.757948 (+ 571us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:35:53.757954 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:35:53.758305 (+ 351us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:35:53.758810 (+ 505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:35:53.758832 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:35:53.759687 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:35:53.759689 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:35:53.760558 (+ 869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:35:53.760567 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:35:53.761090 (+ 523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:35:53.761093 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:35:53.761234 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:35:53.761878 (+ 644us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:35:53.761895 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:35:53.763539 (+ 1644us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:35:53.765582 (+ 2043us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:35:53.765588 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:35:53.765607 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:35:53.765875 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:35:53.766182 (+ 307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:35:53.766186 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:35:53.766188 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:35:53.766341 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:35:53.766769 (+ 428us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:35:53.766777 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:35:53.767038 (+ 261us) client_negotiation.cc:769] Sending connection context
0807 18:35:53.767246 (+ 208us) client_negotiation.cc:240] Negotiation successful
0807 18:35:53.767604 (+ 358us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":122,"threads_started":1}
I20260807 18:35:53.769775 26399 heartbeater.cc:347] Connected to a master server at 127.11.14.126:41615
I20260807 18:35:53.770067 26399 heartbeater.cc:464] Registering TS with master...
I20260807 18:35:53.770731 26399 heartbeater.cc:511] Master 127.11.14.126:41615 requested a full tablet report, sending...
I20260807 18:35:53.772604 26184 ts_manager.cc:194] Registered new tserver with Master: e015739a99844c14829e0b989348e2f1 (127.11.14.65:38891)
I20260807 18:35:53.774312 26184 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.14.65@KRBTEST.COM'} at 127.11.14.65:60963
WARNING: no policy specified for kudu/127.11.14.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.14.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.14.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.66.keytab.
Entry for principal kudu/127.11.14.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.66.keytab.
I20260807 18:35:53.798580 11321 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkz56XN/build/debug/bin/kudu
/tmp/dist-test-taskkz56XN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.14.66:0
--local_ip_for_outbound_sockets=127.11.14.66
--webserver_interface=127.11.14.66
--webserver_port=0
--tserver_master_addrs=127.11.14.126:41615
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.66.keytab
--principal=kudu/127.11.14.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.14.84:39981
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:35:53.918321 26404 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:35:53.918668 26404 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:35:53.918742 26404 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:35:53.922928 26404 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:35:53.923022 26404 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:35:53.923116 26404 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.14.66
I20260807 18:35:53.928524 26404 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.14.84:39981
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.66.keytab
--principal=kudu/127.11.14.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.14.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.14.66
--webserver_port=0
--tserver_master_addrs=127.11.14.126:41615
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26404
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.14.66
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:35:53.929706 26404 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:35:53.930825 26404 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:35:53.940924 26409 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:35:53.940907 26413 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:35:53.940912 26410 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:35:53.941206 26404 server_base.cc:1034] running on GCE node
I20260807 18:35:53.941689 26404 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:35:53.942366 26404 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:35:53.943629 26404 hybrid_clock.cc:648] HybridClock initialized: now 1786127753943582 us; error 54 us; skew 500 ppm
Aug 07 18:35:53 dist-test-slave-6k22 krb5kdc[26133](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127753, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:35:53.946497 26404 init.cc:377] Logged in from keytab as kudu/127.11.14.66@KRBTEST.COM (short username kudu)
I20260807 18:35:53.947448 26404 webserver.cc:460] Webserver started at http://127.11.14.66:42919/ using document root <none> and password file <none>
I20260807 18:35:53.948082 26404 fs_manager.cc:359] Metadata directory not provided
I20260807 18:35:53.948177 26404 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:35:53.948427 26404 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:35:53.950830 26404 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/minicluster-data/ts-1/data/instance:
uuid: "5e4cd084ed8d4729827ac3a8507c961d"
format_stamp: "Formatted at 2026-08-07 18:35:53 on dist-test-slave-6k22"
server_key: "51256bb2d4887aeb74361bf8f1991c3b"
server_key_iv: "ab932955dd02add51fe6556f779490e1"
server_key_version: "encryptionkey@0"
I20260807 18:35:53.951473 26404 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/minicluster-data/ts-1/wal/instance:
uuid: "5e4cd084ed8d4729827ac3a8507c961d"
format_stamp: "Formatted at 2026-08-07 18:35:53 on dist-test-slave-6k22"
server_key: "51256bb2d4887aeb74361bf8f1991c3b"
server_key_iv: "ab932955dd02add51fe6556f779490e1"
server_key_version: "encryptionkey@0"
I20260807 18:35:53.955912 26404 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 18:35:53.958453 26421 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:35:53.959898 26404 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:35:53.960048 26404 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/minicluster-data/ts-1/data,/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/minicluster-data/ts-1/wal
uuid: "5e4cd084ed8d4729827ac3a8507c961d"
format_stamp: "Formatted at 2026-08-07 18:35:53 on dist-test-slave-6k22"
server_key: "51256bb2d4887aeb74361bf8f1991c3b"
server_key_iv: "ab932955dd02add51fe6556f779490e1"
server_key_version: "encryptionkey@0"
I20260807 18:35:53.960176 26404 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:35:53.979379 26404 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:35:53.982568 26404 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:35:53.982854 26404 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:35:53.983516 26404 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:35:53.984503 26404 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:35:53.984587 26404 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:35:53.984679 26404 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:35:53.984725 26404 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:35:53.994979 26404 rpc_server.cc:307] RPC server started. Bound to: 127.11.14.66:39165
I20260807 18:35:53.995004 26549 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.14.66:39165 every 8 connection(s)
I20260807 18:35:53.996034 26404 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/minicluster-data/ts-1/data/info.pb
I20260807 18:35:53.996238 11321 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkz56XN/build/debug/bin/kudu as pid 26404
I20260807 18:35:53.996367 11321 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/minicluster-data/ts-1/wal/instance
I20260807 18:35:53.996711 11321 external_mini_cluster.cc:1428] Setting key 7b0f4198fea250c15e1c31d2dbb33611
Aug 07 18:35:54 dist-test-slave-6k22 krb5kdc[26133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127753, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.66@KRBTEST.COM for kudu/127.11.14.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.11.14.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.14.67@KRBTEST.COM" created.
I20260807 18:35:54.011791 26225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:35:53.998214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.66:53945 (local address 127.11.14.126:41615)
0807 18:35:53.998366 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0807 18:35:53.998370 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:35:53.999431 (+ 1061us) server_negotiation.cc:407] Connection header received
0807 18:35:54.001413 (+ 1982us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:35:54.001416 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:35:54.001476 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:35:54.001588 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:35:54.002802 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:35:54.003337 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:35:54.004773 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:35:54.005010 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:35:54.005135 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:35:54.007781 (+ 2646us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:35:54.007786 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:35:54.007789 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:35:54.007824 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:35:54.009343 (+ 1519us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:35:54.009848 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:35:54.009851 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:35:54.009852 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:35:54.009934 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:35:54.010349 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:35:54.010352 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:35:54.010353 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:35:54.010543 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:35:54.010633 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0807 18:35:54.011285 (+ 652us) server_negotiation.cc:299] Negotiation successful
0807 18:35:54.011442 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260807 18:35:54.012749 26552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:35:53.998694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:41615 (local address 127.11.14.66:53945)
0807 18:35:53.999259 (+ 565us) negotiation.cc:107] Waiting for socket to connect
0807 18:35:53.999331 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0807 18:35:54.001169 (+ 1838us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:35:54.001772 (+ 603us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:35:54.001779 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:35:54.002142 (+ 363us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:35:54.002648 (+ 506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:35:54.002661 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:35:54.003478 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:35:54.003483 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:35:54.004635 (+ 1152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:35:54.004646 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:35:54.005182 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:35:54.005186 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:35:54.005364 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:35:54.006015 (+ 651us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:35:54.006032 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:35:54.007536 (+ 1504us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:35:54.009460 (+ 1924us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:35:54.009465 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:35:54.009477 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:35:54.009730 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:35:54.010032 (+ 302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:35:54.010034 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:35:54.010035 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:35:54.010190 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:35:54.010638 (+ 448us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:35:54.010644 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:35:54.010944 (+ 300us) client_negotiation.cc:769] Sending connection context
0807 18:35:54.011183 (+ 239us) client_negotiation.cc:240] Negotiation successful
0807 18:35:54.011551 (+ 368us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":287,"thread_start_us":123,"threads_started":1}
I20260807 18:35:54.014194 26550 heartbeater.cc:347] Connected to a master server at 127.11.14.126:41615
I20260807 18:35:54.014494 26550 heartbeater.cc:464] Registering TS with master...
I20260807 18:35:54.015173 26550 heartbeater.cc:511] Master 127.11.14.126:41615 requested a full tablet report, sending...
I20260807 18:35:54.016467 26184 ts_manager.cc:194] Registered new tserver with Master: 5e4cd084ed8d4729827ac3a8507c961d (127.11.14.66:39165)
I20260807 18:35:54.017261 26184 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.14.66@KRBTEST.COM'} at 127.11.14.66:53945
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.14.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.67.keytab.
Entry for principal kudu/127.11.14.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.67.keytab.
I20260807 18:35:54.032186 11321 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkz56XN/build/debug/bin/kudu
/tmp/dist-test-taskkz56XN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.14.67:0
--local_ip_for_outbound_sockets=127.11.14.67
--webserver_interface=127.11.14.67
--webserver_port=0
--tserver_master_addrs=127.11.14.126:41615
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.67.keytab
--principal=kudu/127.11.14.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.14.84:39981
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:35:54.147104 26555 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:35:54.147442 26555 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:35:54.147521 26555 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:35:54.151532 26555 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:35:54.151707 26555 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:35:54.151801 26555 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.14.67
I20260807 18:35:54.157061 26555 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.14.84:39981
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.67.keytab
--principal=kudu/127.11.14.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.14.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.14.67
--webserver_port=0
--tserver_master_addrs=127.11.14.126:41615
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26555
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.14.67
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:35:54.158176 26555 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:35:54.159235 26555 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:35:54.169081 26563 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:35:54.169139 26560 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:35:54.169085 26561 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:35:54.169375 26555 server_base.cc:1034] running on GCE node
I20260807 18:35:54.169866 26555 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:35:54.170500 26555 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:35:54.171723 26555 hybrid_clock.cc:648] HybridClock initialized: now 1786127754171700 us; error 26 us; skew 500 ppm
Aug 07 18:35:54 dist-test-slave-6k22 krb5kdc[26133](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127754, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:35:54.174876 26555 init.cc:377] Logged in from keytab as kudu/127.11.14.67@KRBTEST.COM (short username kudu)
I20260807 18:35:54.175906 26555 webserver.cc:460] Webserver started at http://127.11.14.67:33485/ using document root <none> and password file <none>
I20260807 18:35:54.176765 26555 fs_manager.cc:359] Metadata directory not provided
I20260807 18:35:54.176877 26555 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:35:54.177151 26555 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:35:54.180163 26555 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/minicluster-data/ts-2/data/instance:
uuid: "3981583d3760440d863c0aa8b1bfb361"
format_stamp: "Formatted at 2026-08-07 18:35:54 on dist-test-slave-6k22"
server_key: "14005ea34632b5c69f2de8014c948c38"
server_key_iv: "9eec48576c193ae9e9b9ec8ec50e0d22"
server_key_version: "encryptionkey@0"
I20260807 18:35:54.180820 26555 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/minicluster-data/ts-2/wal/instance:
uuid: "3981583d3760440d863c0aa8b1bfb361"
format_stamp: "Formatted at 2026-08-07 18:35:54 on dist-test-slave-6k22"
server_key: "14005ea34632b5c69f2de8014c948c38"
server_key_iv: "9eec48576c193ae9e9b9ec8ec50e0d22"
server_key_version: "encryptionkey@0"
I20260807 18:35:54.184863 26555 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.003s
I20260807 18:35:54.187309 26570 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:35:54.188655 26555 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260807 18:35:54.188788 26555 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/minicluster-data/ts-2/data,/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/minicluster-data/ts-2/wal
uuid: "3981583d3760440d863c0aa8b1bfb361"
format_stamp: "Formatted at 2026-08-07 18:35:54 on dist-test-slave-6k22"
server_key: "14005ea34632b5c69f2de8014c948c38"
server_key_iv: "9eec48576c193ae9e9b9ec8ec50e0d22"
server_key_version: "encryptionkey@0"
I20260807 18:35:54.188897 26555 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:35:54.201771 26555 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:35:54.205142 26555 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:35:54.205356 26555 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:35:54.205989 26555 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:35:54.206837 26555 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:35:54.206892 26555 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:35:54.206976 26555 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:35:54.207013 26555 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:35:54.216419 26555 rpc_server.cc:307] RPC server started. Bound to: 127.11.14.67:35171
I20260807 18:35:54.216445 26694 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.14.67:35171 every 8 connection(s)
I20260807 18:35:54.217338 26555 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/minicluster-data/ts-2/data/info.pb
I20260807 18:35:54.219759 11321 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkz56XN/build/debug/bin/kudu as pid 26555
I20260807 18:35:54.219863 11321 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/minicluster-data/ts-2/wal/instance
I20260807 18:35:54.220244 11321 external_mini_cluster.cc:1428] Setting key 3e2a74896c189fecb507c22b66bea612
Aug 07 18:35:54 dist-test-slave-6k22 krb5kdc[26133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127754, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.67@KRBTEST.COM for kudu/127.11.14.126@KRBTEST.COM
I20260807 18:35:54.231957 26225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:35:54.219355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.67:54039 (local address 127.11.14.126:41615)
0807 18:35:54.219569 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0807 18:35:54.219576 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:35:54.220626 (+ 1050us) server_negotiation.cc:407] Connection header received
0807 18:35:54.222549 (+ 1923us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:35:54.222553 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:35:54.222617 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:35:54.222700 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:35:54.224054 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:35:54.224585 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:35:54.225683 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:35:54.225890 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:35:54.225965 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:35:54.228398 (+ 2433us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:35:54.228401 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:35:54.228403 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:35:54.228429 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:35:54.229923 (+ 1494us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:35:54.230423 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:35:54.230426 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:35:54.230427 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:35:54.230506 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:35:54.230838 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:35:54.230841 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:35:54.230842 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:35:54.231030 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:35:54.231111 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0807 18:35:54.231716 (+ 605us) server_negotiation.cc:299] Negotiation successful
0807 18:35:54.231826 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260807 18:35:54.232736 26697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:35:54.219841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:41615 (local address 127.11.14.67:54039)
0807 18:35:54.220434 (+ 593us) negotiation.cc:107] Waiting for socket to connect
0807 18:35:54.220497 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0807 18:35:54.222306 (+ 1809us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:35:54.222924 (+ 618us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:35:54.222931 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:35:54.223331 (+ 400us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:35:54.223918 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:35:54.223933 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:35:54.224698 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:35:54.224700 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:35:54.225573 (+ 873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:35:54.225582 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:35:54.226025 (+ 443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:35:54.226028 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:35:54.226170 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:35:54.226710 (+ 540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:35:54.226726 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:35:54.228247 (+ 1521us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:35:54.230051 (+ 1804us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:35:54.230055 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:35:54.230069 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:35:54.230312 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:35:54.230591 (+ 279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:35:54.230593 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:35:54.230594 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:35:54.230725 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:35:54.231126 (+ 401us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:35:54.231131 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:35:54.231362 (+ 231us) client_negotiation.cc:769] Sending connection context
0807 18:35:54.231511 (+ 149us) client_negotiation.cc:240] Negotiation successful
0807 18:35:54.231857 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":328,"thread_start_us":127,"threads_started":1}
I20260807 18:35:54.234004 26695 heartbeater.cc:347] Connected to a master server at 127.11.14.126:41615
I20260807 18:35:54.234292 26695 heartbeater.cc:464] Registering TS with master...
I20260807 18:35:54.234838 26695 heartbeater.cc:511] Master 127.11.14.126:41615 requested a full tablet report, sending...
I20260807 18:35:54.235885 26184 ts_manager.cc:194] Registered new tserver with Master: 3981583d3760440d863c0aa8b1bfb361 (127.11.14.67:35171)
I20260807 18:35:54.236428 26184 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.14.67@KRBTEST.COM'} at 127.11.14.67:54039
I20260807 18:35:54.247015 11321 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 18:35:54.257593 26225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:35:54.249350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44790 (local address 127.11.14.126:41615)
0807 18:35:54.249499 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0807 18:35:54.249504 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:35:54.249729 (+ 225us) server_negotiation.cc:407] Connection header received
0807 18:35:54.249861 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:35:54.249863 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:35:54.249906 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:35:54.249977 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:35:54.250767 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:35:54.251275 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:35:54.252354 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:35:54.252594 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:35:54.252698 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:35:54.254156 (+ 1458us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:35:54.254159 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:35:54.254161 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:35:54.254185 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:35:54.255835 (+ 1650us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:35:54.256297 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:35:54.256300 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:35:54.256301 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:35:54.256375 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:35:54.256679 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:35:54.256681 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:35:54.256682 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:35:54.256906 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:35:54.257010 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0807 18:35:54.257299 (+ 289us) server_negotiation.cc:299] Negotiation successful
0807 18:35:54.257396 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260807 18:35:54.260715 26184 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44790:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 18:35:54.263864 26184 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 18:35:54.279675 26710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:35:54.273473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.65:38891 (local address 127.0.0.1:39388)
0807 18:35:54.274348 (+ 875us) negotiation.cc:107] Waiting for socket to connect
0807 18:35:54.274393 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 18:35:54.274519 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:35:54.275211 (+ 692us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:35:54.275217 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:35:54.275257 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:35:54.275458 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:35:54.275466 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:35:54.276995 (+ 1529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:35:54.276998 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:35:54.278158 (+ 1160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:35:54.278169 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:35:54.279171 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:35:54.279174 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:35:54.279260 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:35:54.279293 (+ 33us) client_negotiation.cc:769] Sending connection context
0807 18:35:54.279382 (+ 89us) client_negotiation.cc:240] Negotiation successful
0807 18:35:54.279472 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":749,"mutex_wait_us":64,"spinlock_wait_cycles":7936,"thread_start_us":99,"threads_started":1}
I20260807 18:35:54.279754 26712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:35:54.273840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39388 (local address 127.11.14.65:38891)
0807 18:35:54.274695 (+ 855us) server_negotiation.cc:206] Beginning negotiation
0807 18:35:54.274702 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:35:54.274725 (+ 23us) server_negotiation.cc:407] Connection header received
0807 18:35:54.274806 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:35:54.274811 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:35:54.275041 (+ 230us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:35:54.275337 (+ 296us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:35:54.275613 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:35:54.276862 (+ 1249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:35:54.278318 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:35:54.279033 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:35:54.279157 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:35:54.279336 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0807 18:35:54.279444 (+ 108us) server_negotiation.cc:299] Negotiation successful
0807 18:35:54.279576 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":743,"thread_start_us":78,"threads_started":1}
I20260807 18:35:54.281531 26713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:35:54.274234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.67:35171 (local address 127.0.0.1:54630)
0807 18:35:54.274814 (+ 580us) negotiation.cc:107] Waiting for socket to connect
0807 18:35:54.274827 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 18:35:54.274902 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:35:54.275669 (+ 767us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:35:54.275672 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:35:54.275689 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:35:54.275846 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:35:54.275855 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:35:54.277072 (+ 1217us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:35:54.277076 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:35:54.278682 (+ 1606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:35:54.278694 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:35:54.281132 (+ 2438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:35:54.281135 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:35:54.281222 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:35:54.281246 (+ 24us) client_negotiation.cc:769] Sending connection context
0807 18:35:54.281331 (+ 85us) client_negotiation.cc:240] Negotiation successful
0807 18:35:54.281400 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":526,"mutex_wait_us":206,"thread_start_us":119,"threads_started":1}
I20260807 18:35:54.281683 26714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:35:54.274753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54630 (local address 127.11.14.67:35171)
0807 18:35:54.275165 (+ 412us) server_negotiation.cc:206] Beginning negotiation
0807 18:35:54.275178 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 18:35:54.275197 (+ 19us) server_negotiation.cc:407] Connection header received
0807 18:35:54.275267 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:35:54.275273 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:35:54.275460 (+ 187us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:35:54.275673 (+ 213us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:35:54.275995 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:35:54.276908 (+ 913us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:35:54.280140 (+ 3232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:35:54.280993 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:35:54.281109 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:35:54.281213 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0807 18:35:54.281394 (+ 181us) server_negotiation.cc:299] Negotiation successful
0807 18:35:54.281511 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":294,"thread_start_us":138,"threads_started":1}
I20260807 18:35:54.281736 26709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:35:54.273275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.66:39165 (local address 127.0.0.1:38516)
0807 18:35:54.274332 (+ 1057us) negotiation.cc:107] Waiting for socket to connect
0807 18:35:54.274390 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0807 18:35:54.274521 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:35:54.275213 (+ 692us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:35:54.275220 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:35:54.275259 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:35:54.275787 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:35:54.275796 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:35:54.277156 (+ 1360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:35:54.277159 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:35:54.278682 (+ 1523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:35:54.278694 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:35:54.281275 (+ 2581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:35:54.281278 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:35:54.281407 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:35:54.281434 (+ 27us) client_negotiation.cc:769] Sending connection context
0807 18:35:54.281517 (+ 83us) client_negotiation.cc:240] Negotiation successful
0807 18:35:54.281589 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":862,"mutex_wait_us":127,"spinlock_wait_cycles":2048,"thread_start_us":111,"threads_started":1}
I20260807 18:35:54.282210 26711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:35:54.273439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38516 (local address 127.11.14.66:39165)
0807 18:35:54.274130 (+ 691us) server_negotiation.cc:206] Beginning negotiation
0807 18:35:54.274144 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 18:35:54.274676 (+ 532us) server_negotiation.cc:407] Connection header received
0807 18:35:54.274770 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:35:54.274776 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:35:54.275007 (+ 231us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:35:54.275233 (+ 226us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:35:54.275925 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:35:54.277013 (+ 1088us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:35:54.278833 (+ 1820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:35:54.279735 (+ 902us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:35:54.279861 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:35:54.280055 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0807 18:35:54.281921 (+ 1866us) server_negotiation.cc:299] Negotiation successful
0807 18:35:54.282031 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":578,"thread_start_us":153,"threads_started":1}
I20260807 18:35:54.283471 26327 tablet_service.cc:1467] Processing CreateTablet for tablet 474f2cd763444a55a47c905282cde92f (DEFAULT_TABLE table=test-table [id=7673f3c8278b4969abde6b453856b13e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:35:54.284806 26327 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 474f2cd763444a55a47c905282cde92f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:35:54.284515 26624 tablet_service.cc:1467] Processing CreateTablet for tablet 474f2cd763444a55a47c905282cde92f (DEFAULT_TABLE table=test-table [id=7673f3c8278b4969abde6b453856b13e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:35:54.285140 26474 tablet_service.cc:1467] Processing CreateTablet for tablet 474f2cd763444a55a47c905282cde92f (DEFAULT_TABLE table=test-table [id=7673f3c8278b4969abde6b453856b13e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:35:54.285579 26624 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 474f2cd763444a55a47c905282cde92f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:35:54.286099 26474 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 474f2cd763444a55a47c905282cde92f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:35:54.292377 26715 tablet_bootstrap.cc:492] T 474f2cd763444a55a47c905282cde92f P 5e4cd084ed8d4729827ac3a8507c961d: Bootstrap starting.
I20260807 18:35:54.293825 26717 tablet_bootstrap.cc:492] T 474f2cd763444a55a47c905282cde92f P 3981583d3760440d863c0aa8b1bfb361: Bootstrap starting.
I20260807 18:35:54.295722 26715 tablet_bootstrap.cc:654] T 474f2cd763444a55a47c905282cde92f P 5e4cd084ed8d4729827ac3a8507c961d: Neither blocks nor log segments found. Creating new log.
I20260807 18:35:54.295924 26716 tablet_bootstrap.cc:492] T 474f2cd763444a55a47c905282cde92f P e015739a99844c14829e0b989348e2f1: Bootstrap starting.
I20260807 18:35:54.296550 26717 tablet_bootstrap.cc:654] T 474f2cd763444a55a47c905282cde92f P 3981583d3760440d863c0aa8b1bfb361: Neither blocks nor log segments found. Creating new log.
I20260807 18:35:54.296729 26715 log.cc:824] T 474f2cd763444a55a47c905282cde92f P 5e4cd084ed8d4729827ac3a8507c961d: Log is configured to *not* fsync() on all Append() calls
I20260807 18:35:54.297442 26717 log.cc:824] T 474f2cd763444a55a47c905282cde92f P 3981583d3760440d863c0aa8b1bfb361: Log is configured to *not* fsync() on all Append() calls
I20260807 18:35:54.298192 26716 tablet_bootstrap.cc:654] T 474f2cd763444a55a47c905282cde92f P e015739a99844c14829e0b989348e2f1: Neither blocks nor log segments found. Creating new log.
I20260807 18:35:54.299057 26715 tablet_bootstrap.cc:492] T 474f2cd763444a55a47c905282cde92f P 5e4cd084ed8d4729827ac3a8507c961d: No bootstrap required, opened a new log
I20260807 18:35:54.299088 26716 log.cc:824] T 474f2cd763444a55a47c905282cde92f P e015739a99844c14829e0b989348e2f1: Log is configured to *not* fsync() on all Append() calls
I20260807 18:35:54.299300 26715 ts_tablet_manager.cc:1397] T 474f2cd763444a55a47c905282cde92f P 5e4cd084ed8d4729827ac3a8507c961d: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260807 18:35:54.299685 26717 tablet_bootstrap.cc:492] T 474f2cd763444a55a47c905282cde92f P 3981583d3760440d863c0aa8b1bfb361: No bootstrap required, opened a new log
I20260807 18:35:54.299928 26717 ts_tablet_manager.cc:1397] T 474f2cd763444a55a47c905282cde92f P 3981583d3760440d863c0aa8b1bfb361: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260807 18:35:54.301200 26716 tablet_bootstrap.cc:492] T 474f2cd763444a55a47c905282cde92f P e015739a99844c14829e0b989348e2f1: No bootstrap required, opened a new log
I20260807 18:35:54.301406 26716 ts_tablet_manager.cc:1397] T 474f2cd763444a55a47c905282cde92f P e015739a99844c14829e0b989348e2f1: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260807 18:35:54.303426 26715 raft_consensus.cc:348] T 474f2cd763444a55a47c905282cde92f P 5e4cd084ed8d4729827ac3a8507c961d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e4cd084ed8d4729827ac3a8507c961d" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 39165 } } peers { permanent_uuid: "e015739a99844c14829e0b989348e2f1" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 38891 } } peers { permanent_uuid: "3981583d3760440d863c0aa8b1bfb361" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 35171 } }
I20260807 18:35:54.303762 26715 raft_consensus.cc:374] T 474f2cd763444a55a47c905282cde92f P 5e4cd084ed8d4729827ac3a8507c961d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:35:54.303877 26715 raft_consensus.cc:729] T 474f2cd763444a55a47c905282cde92f P 5e4cd084ed8d4729827ac3a8507c961d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e4cd084ed8d4729827ac3a8507c961d, State: Initialized, Role: FOLLOWER
I20260807 18:35:54.304383 26715 consensus_queue.cc:260] T 474f2cd763444a55a47c905282cde92f P 5e4cd084ed8d4729827ac3a8507c961d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e4cd084ed8d4729827ac3a8507c961d" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 39165 } } peers { permanent_uuid: "e015739a99844c14829e0b989348e2f1" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 38891 } } peers { permanent_uuid: "3981583d3760440d863c0aa8b1bfb361" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 35171 } }
I20260807 18:35:54.304345 26717 raft_consensus.cc:348] T 474f2cd763444a55a47c905282cde92f P 3981583d3760440d863c0aa8b1bfb361 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e4cd084ed8d4729827ac3a8507c961d" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 39165 } } peers { permanent_uuid: "e015739a99844c14829e0b989348e2f1" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 38891 } } peers { permanent_uuid: "3981583d3760440d863c0aa8b1bfb361" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 35171 } }
I20260807 18:35:54.304638 26717 raft_consensus.cc:374] T 474f2cd763444a55a47c905282cde92f P 3981583d3760440d863c0aa8b1bfb361 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:35:54.304709 26717 raft_consensus.cc:729] T 474f2cd763444a55a47c905282cde92f P 3981583d3760440d863c0aa8b1bfb361 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3981583d3760440d863c0aa8b1bfb361, State: Initialized, Role: FOLLOWER
I20260807 18:35:54.305142 26550 heartbeater.cc:503] Master 127.11.14.126:41615 was elected leader, sending a full tablet report...
I20260807 18:35:54.305262 26717 consensus_queue.cc:260] T 474f2cd763444a55a47c905282cde92f P 3981583d3760440d863c0aa8b1bfb361 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e4cd084ed8d4729827ac3a8507c961d" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 39165 } } peers { permanent_uuid: "e015739a99844c14829e0b989348e2f1" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 38891 } } peers { permanent_uuid: "3981583d3760440d863c0aa8b1bfb361" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 35171 } }
I20260807 18:35:54.305970 26695 heartbeater.cc:503] Master 127.11.14.126:41615 was elected leader, sending a full tablet report...
I20260807 18:35:54.306319 26717 ts_tablet_manager.cc:1428] T 474f2cd763444a55a47c905282cde92f P 3981583d3760440d863c0aa8b1bfb361: Time spent starting tablet: real 0.006s user 0.001s sys 0.004s
I20260807 18:35:54.306685 26715 ts_tablet_manager.cc:1428] T 474f2cd763444a55a47c905282cde92f P 5e4cd084ed8d4729827ac3a8507c961d: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260807 18:35:54.307018 26716 raft_consensus.cc:348] T 474f2cd763444a55a47c905282cde92f P e015739a99844c14829e0b989348e2f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e4cd084ed8d4729827ac3a8507c961d" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 39165 } } peers { permanent_uuid: "e015739a99844c14829e0b989348e2f1" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 38891 } } peers { permanent_uuid: "3981583d3760440d863c0aa8b1bfb361" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 35171 } }
I20260807 18:35:54.307349 26716 raft_consensus.cc:374] T 474f2cd763444a55a47c905282cde92f P e015739a99844c14829e0b989348e2f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:35:54.307448 26716 raft_consensus.cc:729] T 474f2cd763444a55a47c905282cde92f P e015739a99844c14829e0b989348e2f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e015739a99844c14829e0b989348e2f1, State: Initialized, Role: FOLLOWER
I20260807 18:35:54.307938 26716 consensus_queue.cc:260] T 474f2cd763444a55a47c905282cde92f P e015739a99844c14829e0b989348e2f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e4cd084ed8d4729827ac3a8507c961d" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 39165 } } peers { permanent_uuid: "e015739a99844c14829e0b989348e2f1" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 38891 } } peers { permanent_uuid: "3981583d3760440d863c0aa8b1bfb361" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 35171 } }
I20260807 18:35:54.308607 26399 heartbeater.cc:503] Master 127.11.14.126:41615 was elected leader, sending a full tablet report...
I20260807 18:35:54.308995 26716 ts_tablet_manager.cc:1428] T 474f2cd763444a55a47c905282cde92f P e015739a99844c14829e0b989348e2f1: Time spent starting tablet: real 0.008s user 0.007s sys 0.000s
I20260807 18:35:54.400853 26724 raft_consensus.cc:482] T 474f2cd763444a55a47c905282cde92f P 3981583d3760440d863c0aa8b1bfb361 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:35:54.401121 26724 raft_consensus.cc:504] T 474f2cd763444a55a47c905282cde92f P 3981583d3760440d863c0aa8b1bfb361 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e4cd084ed8d4729827ac3a8507c961d" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 39165 } } peers { permanent_uuid: "e015739a99844c14829e0b989348e2f1" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 38891 } } peers { permanent_uuid: "3981583d3760440d863c0aa8b1bfb361" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 35171 } }
I20260807 18:35:54.402451 26724 leader_election.cc:288] T 474f2cd763444a55a47c905282cde92f P 3981583d3760440d863c0aa8b1bfb361 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e4cd084ed8d4729827ac3a8507c961d (127.11.14.66:39165), e015739a99844c14829e0b989348e2f1 (127.11.14.65:38891)
I20260807 18:35:54.406924 26711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:35:54.402943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.67:39599 (local address 127.11.14.66:39165)
0807 18:35:54.403100 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0807 18:35:54.403107 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:35:54.403152 (+ 45us) server_negotiation.cc:407] Connection header received
0807 18:35:54.403218 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:35:54.403222 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:35:54.403279 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:35:54.403421 (+ 142us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:35:54.403780 (+ 359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:35:54.404456 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:35:54.405735 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:35:54.406388 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:35:54.406483 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:35:54.406526 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0807 18:35:54.406760 (+ 234us) server_negotiation.cc:299] Negotiation successful
0807 18:35:54.406820 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260807 18:35:54.406968 26697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:35:54.402778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.66:39165 (local address 127.11.14.67:39599)
0807 18:35:54.402935 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0807 18:35:54.402953 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 18:35:54.403068 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:35:54.403430 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:35:54.403433 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:35:54.403451 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:35:54.403639 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:35:54.403648 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:35:54.404573 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:35:54.404576 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:35:54.405593 (+ 1017us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:35:54.405603 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:35:54.406507 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:35:54.406511 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:35:54.406621 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:35:54.406653 (+ 32us) client_negotiation.cc:769] Sending connection context
0807 18:35:54.406732 (+ 79us) client_negotiation.cc:240] Negotiation successful
0807 18:35:54.406806 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57,"mutex_wait_us":18}
I20260807 18:35:54.407670 26726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:35:54.402778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.65:38891 (local address 127.11.14.67:51669)
0807 18:35:54.403389 (+ 611us) negotiation.cc:107] Waiting for socket to connect
0807 18:35:54.403400 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 18:35:54.403522 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:35:54.403911 (+ 389us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:35:54.403914 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:35:54.403932 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:35:54.404051 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:35:54.404058 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:35:54.405122 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:35:54.405125 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:35:54.406185 (+ 1060us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:35:54.406196 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:35:54.407212 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:35:54.407216 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:35:54.407306 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:35:54.407332 (+ 26us) client_negotiation.cc:769] Sending connection context
0807 18:35:54.407438 (+ 106us) client_negotiation.cc:240] Negotiation successful
0807 18:35:54.407505 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":553,"mutex_wait_us":277,"thread_start_us":140,"threads_started":1}
I20260807 18:35:54.407680 26712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:35:54.402902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.67:51669 (local address 127.11.14.65:38891)
0807 18:35:54.403030 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0807 18:35:54.403035 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:35:54.403490 (+ 455us) server_negotiation.cc:407] Connection header received
0807 18:35:54.403740 (+ 250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:35:54.403746 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:35:54.403805 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:35:54.403866 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:35:54.404222 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:35:54.404964 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:35:54.406327 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:35:54.406957 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:35:54.407076 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:35:54.407132 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0807 18:35:54.407458 (+ 326us) server_negotiation.cc:299] Negotiation successful
0807 18:35:54.407537 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260807 18:35:54.407692 26494 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "474f2cd763444a55a47c905282cde92f" candidate_uuid: "3981583d3760440d863c0aa8b1bfb361" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e4cd084ed8d4729827ac3a8507c961d" is_pre_election: true
I20260807 18:35:54.408114 26494 raft_consensus.cc:2454] T 474f2cd763444a55a47c905282cde92f P 5e4cd084ed8d4729827ac3a8507c961d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3981583d3760440d863c0aa8b1bfb361 in term 0.
I20260807 18:35:54.408396 26349 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "474f2cd763444a55a47c905282cde92f" candidate_uuid: "3981583d3760440d863c0aa8b1bfb361" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e015739a99844c14829e0b989348e2f1" is_pre_election: true
I20260807 18:35:54.408694 26349 raft_consensus.cc:2454] T 474f2cd763444a55a47c905282cde92f P e015739a99844c14829e0b989348e2f1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3981583d3760440d863c0aa8b1bfb361 in term 0.
I20260807 18:35:54.408692 26573 leader_election.cc:302] T 474f2cd763444a55a47c905282cde92f P 3981583d3760440d863c0aa8b1bfb361 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3981583d3760440d863c0aa8b1bfb361, 5e4cd084ed8d4729827ac3a8507c961d; no voters:
I20260807 18:35:54.409034 26724 raft_consensus.cc:2781] T 474f2cd763444a55a47c905282cde92f P 3981583d3760440d863c0aa8b1bfb361 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:35:54.409137 26724 raft_consensus.cc:482] T 474f2cd763444a55a47c905282cde92f P 3981583d3760440d863c0aa8b1bfb361 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:35:54.409204 26724 raft_consensus.cc:3037] T 474f2cd763444a55a47c905282cde92f P 3981583d3760440d863c0aa8b1bfb361 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:35:54.410276 26724 raft_consensus.cc:504] T 474f2cd763444a55a47c905282cde92f P 3981583d3760440d863c0aa8b1bfb361 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e4cd084ed8d4729827ac3a8507c961d" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 39165 } } peers { permanent_uuid: "e015739a99844c14829e0b989348e2f1" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 38891 } } peers { permanent_uuid: "3981583d3760440d863c0aa8b1bfb361" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 35171 } }
I20260807 18:35:54.410676 26724 leader_election.cc:288] T 474f2cd763444a55a47c905282cde92f P 3981583d3760440d863c0aa8b1bfb361 [CANDIDATE]: Term 1 election: Requested vote from peers 5e4cd084ed8d4729827ac3a8507c961d (127.11.14.66:39165), e015739a99844c14829e0b989348e2f1 (127.11.14.65:38891)
I20260807 18:35:54.411041 26349 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "474f2cd763444a55a47c905282cde92f" candidate_uuid: "3981583d3760440d863c0aa8b1bfb361" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e015739a99844c14829e0b989348e2f1"
I20260807 18:35:54.411032 26494 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "474f2cd763444a55a47c905282cde92f" candidate_uuid: "3981583d3760440d863c0aa8b1bfb361" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e4cd084ed8d4729827ac3a8507c961d"
I20260807 18:35:54.411162 26494 raft_consensus.cc:3037] T 474f2cd763444a55a47c905282cde92f P 5e4cd084ed8d4729827ac3a8507c961d [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:35:54.411175 26349 raft_consensus.cc:3037] T 474f2cd763444a55a47c905282cde92f P e015739a99844c14829e0b989348e2f1 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:35:54.412562 26494 raft_consensus.cc:2454] T 474f2cd763444a55a47c905282cde92f P 5e4cd084ed8d4729827ac3a8507c961d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3981583d3760440d863c0aa8b1bfb361 in term 1.
I20260807 18:35:54.412562 26349 raft_consensus.cc:2454] T 474f2cd763444a55a47c905282cde92f P e015739a99844c14829e0b989348e2f1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3981583d3760440d863c0aa8b1bfb361 in term 1.
I20260807 18:35:54.412863 26571 leader_election.cc:302] T 474f2cd763444a55a47c905282cde92f P 3981583d3760440d863c0aa8b1bfb361 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3981583d3760440d863c0aa8b1bfb361, e015739a99844c14829e0b989348e2f1; no voters:
I20260807 18:35:54.413043 26724 raft_consensus.cc:2781] T 474f2cd763444a55a47c905282cde92f P 3981583d3760440d863c0aa8b1bfb361 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:35:54.413301 26724 raft_consensus.cc:686] T 474f2cd763444a55a47c905282cde92f P 3981583d3760440d863c0aa8b1bfb361 [term 1 LEADER]: Becoming Leader. State: Replica: 3981583d3760440d863c0aa8b1bfb361, State: Running, Role: LEADER
I20260807 18:35:54.413697 26724 consensus_queue.cc:237] T 474f2cd763444a55a47c905282cde92f P 3981583d3760440d863c0aa8b1bfb361 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e4cd084ed8d4729827ac3a8507c961d" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 39165 } } peers { permanent_uuid: "e015739a99844c14829e0b989348e2f1" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 38891 } } peers { permanent_uuid: "3981583d3760440d863c0aa8b1bfb361" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 35171 } }
I20260807 18:35:54.417047 26184 catalog_manager.cc:5495] T 474f2cd763444a55a47c905282cde92f P 3981583d3760440d863c0aa8b1bfb361 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3981583d3760440d863c0aa8b1bfb361 (127.11.14.67). New cstate: current_term: 1 leader_uuid: "3981583d3760440d863c0aa8b1bfb361" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e4cd084ed8d4729827ac3a8507c961d" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 39165 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e015739a99844c14829e0b989348e2f1" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 38891 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3981583d3760440d863c0aa8b1bfb361" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 35171 } health_report { overall_health: HEALTHY } } }
I20260807 18:35:54.435065 26714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:35:54.430657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54634 (local address 127.11.14.67:35171)
0807 18:35:54.430800 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0807 18:35:54.430804 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:35:54.430819 (+ 15us) server_negotiation.cc:407] Connection header received
0807 18:35:54.430949 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:35:54.430951 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:35:54.431020 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:35:54.431108 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:35:54.431435 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:35:54.432183 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:35:54.433057 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:35:54.433290 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:35:54.433378 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:35:54.433702 (+ 324us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:35:54.434352 (+ 650us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:35:54.434490 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0807 18:35:54.434836 (+ 346us) server_negotiation.cc:299] Negotiation successful
0807 18:35:54.434902 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260807 18:35:54.441789 26494 raft_consensus.cc:1261] T 474f2cd763444a55a47c905282cde92f P 5e4cd084ed8d4729827ac3a8507c961d [term 1 FOLLOWER]: Refusing update from remote peer 3981583d3760440d863c0aa8b1bfb361: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:35:54.441838 26349 raft_consensus.cc:1261] T 474f2cd763444a55a47c905282cde92f P e015739a99844c14829e0b989348e2f1 [term 1 FOLLOWER]: Refusing update from remote peer 3981583d3760440d863c0aa8b1bfb361: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:35:54.442581 26727 consensus_queue.cc:1035] T 474f2cd763444a55a47c905282cde92f P 3981583d3760440d863c0aa8b1bfb361 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e4cd084ed8d4729827ac3a8507c961d" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 39165 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:35:54.442832 26724 consensus_queue.cc:1035] T 474f2cd763444a55a47c905282cde92f P 3981583d3760440d863c0aa8b1bfb361 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e015739a99844c14829e0b989348e2f1" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 38891 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:35:54.460110 26184 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44790:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:35:54.460316 26184 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44790:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:35:54.462909 26184 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c582c50c1bb949ab9b83a517755fc867: Sending DeleteTablet for 3 replicas of tablet 474f2cd763444a55a47c905282cde92f
I20260807 18:35:54.463862 26474 tablet_service.cc:1514] Processing DeleteTablet for tablet 474f2cd763444a55a47c905282cde92f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:35:54 UTC) from {username='kudu', principal='kudu/127.11.14.126@KRBTEST.COM'} at 127.0.0.1:38516
I20260807 18:35:54.464514 26739 tablet_replica.cc:330] T 474f2cd763444a55a47c905282cde92f P 5e4cd084ed8d4729827ac3a8507c961d: stopping tablet replica
I20260807 18:35:54.464545 26327 tablet_service.cc:1514] Processing DeleteTablet for tablet 474f2cd763444a55a47c905282cde92f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:35:54 UTC) from {username='kudu', principal='kudu/127.11.14.126@KRBTEST.COM'} at 127.0.0.1:39388
I20260807 18:35:54.465057 26739 raft_consensus.cc:2229] T 474f2cd763444a55a47c905282cde92f P 5e4cd084ed8d4729827ac3a8507c961d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:35:54.465492 26739 raft_consensus.cc:2258] T 474f2cd763444a55a47c905282cde92f P 5e4cd084ed8d4729827ac3a8507c961d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:35:54.465684 26624 tablet_service.cc:1514] Processing DeleteTablet for tablet 474f2cd763444a55a47c905282cde92f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:35:54 UTC) from {username='kudu', principal='kudu/127.11.14.126@KRBTEST.COM'} at 127.0.0.1:54630
I20260807 18:35:54.466187 26741 tablet_replica.cc:330] T 474f2cd763444a55a47c905282cde92f P 3981583d3760440d863c0aa8b1bfb361: stopping tablet replica
I20260807 18:35:54.466544 26741 raft_consensus.cc:2229] T 474f2cd763444a55a47c905282cde92f P 3981583d3760440d863c0aa8b1bfb361 [term 1 LEADER]: Raft consensus shutting down.
I20260807 18:35:54.466845 11321 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkz56XN/build/debug/bin/kudu with pid 26259
I20260807 18:35:54.468511 26741 raft_consensus.cc:2258] T 474f2cd763444a55a47c905282cde92f P 3981583d3760440d863c0aa8b1bfb361 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:35:54.470718 26741 ts_tablet_manager.cc:1905] T 474f2cd763444a55a47c905282cde92f P 3981583d3760440d863c0aa8b1bfb361: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:35:54.474498 26741 ts_tablet_manager.cc:1918] T 474f2cd763444a55a47c905282cde92f P 3981583d3760440d863c0aa8b1bfb361: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:35:54.474677 26741 log.cc:1196] T 474f2cd763444a55a47c905282cde92f P 3981583d3760440d863c0aa8b1bfb361: Deleting WAL directory at /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/minicluster-data/ts-2/wal/wals/474f2cd763444a55a47c905282cde92f
I20260807 18:35:54.475122 26741 ts_tablet_manager.cc:1939] T 474f2cd763444a55a47c905282cde92f P 3981583d3760440d863c0aa8b1bfb361: Deleting consensus metadata
I20260807 18:35:54.475415 26739 ts_tablet_manager.cc:1905] T 474f2cd763444a55a47c905282cde92f P 5e4cd084ed8d4729827ac3a8507c961d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:35:54.476639 26171 catalog_manager.cc:4841] TS 3981583d3760440d863c0aa8b1bfb361 (127.11.14.67:35171): tablet 474f2cd763444a55a47c905282cde92f (table test-table [id=7673f3c8278b4969abde6b453856b13e]) successfully deleted
I20260807 18:35:54.478156 26739 ts_tablet_manager.cc:1918] T 474f2cd763444a55a47c905282cde92f P 5e4cd084ed8d4729827ac3a8507c961d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:35:54.478322 26739 log.cc:1196] T 474f2cd763444a55a47c905282cde92f P 5e4cd084ed8d4729827ac3a8507c961d: Deleting WAL directory at /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/minicluster-data/ts-1/wal/wals/474f2cd763444a55a47c905282cde92f
I20260807 18:35:54.478715 26739 ts_tablet_manager.cc:1939] T 474f2cd763444a55a47c905282cde92f P 5e4cd084ed8d4729827ac3a8507c961d: Deleting consensus metadata
I20260807 18:35:54.480043 26169 catalog_manager.cc:4841] TS 5e4cd084ed8d4729827ac3a8507c961d (127.11.14.66:39165): tablet 474f2cd763444a55a47c905282cde92f (table test-table [id=7673f3c8278b4969abde6b453856b13e]) successfully deleted
W20260807 18:35:54.481941 26167 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.14.65:38891 (error 108)
I20260807 18:35:54.482870 11321 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkz56XN/build/debug/bin/kudu with pid 26404
I20260807 18:35:54.483153 26709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:35:54.482792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.65:38891 (local address 127.0.0.1:39402)
0807 18:35:54.482952 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0807 18:35:54.483051 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.65:38891: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
W20260807 18:35:54.484879 26167 catalog_manager.cc:4584] TS e015739a99844c14829e0b989348e2f1 (127.11.14.65:38891): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 474f2cd763444a55a47c905282cde92f: Network error: Client connection negotiation failed: client connection to 127.11.14.65:38891: connect: Connection refused (error 111)
I20260807 18:35:54.495088 11321 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkz56XN/build/debug/bin/kudu with pid 26555
I20260807 18:35:54.504026 11321 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkz56XN/build/debug/bin/kudu with pid 26150
I20260807 18:35:54.513172 11321 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkz56XN/build/debug/bin/kudu
/tmp/dist-test-taskkz56XN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.14.126:41615
--webserver_interface=127.11.14.126
--webserver_port=44139
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.126.keytab
--principal=kudu/127.11.14.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.14.84:39981
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.14.126:41615
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:35:54.629992 26743 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:35:54.630270 26743 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:35:54.630311 26743 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:35:54.634403 26743 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:35:54.634469 26743 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:35:54.634490 26743 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:35:54.634506 26743 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:35:54.634521 26743 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:35:54.639971 26743 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.14.84:39981
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.11.14.126:41615
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.126.keytab
--principal=kudu/127.11.14.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.14.126:41615
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.14.126
--webserver_port=44139
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26743
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:35:54.641110 26743 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:35:54.642199 26743 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 18:35:54.656848 26743 server_base.cc:1034] running on GCE node
W20260807 18:35:54.656800 26750 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:35:54.656805 26751 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:35:54.656798 26753 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:35:54.657639 26743 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:35:54.658890 26743 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:35:54.660089 26743 hybrid_clock.cc:648] HybridClock initialized: now 1786127754660064 us; error 37 us; skew 500 ppm
Aug 07 18:35:54 dist-test-slave-6k22 krb5kdc[26133](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127754, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:35:54.662923 26743 init.cc:377] Logged in from keytab as kudu/127.11.14.126@KRBTEST.COM (short username kudu)
I20260807 18:35:54.663991 26743 webserver.cc:460] Webserver started at http://127.11.14.126:44139/ using document root <none> and password file <none>
I20260807 18:35:54.664577 26743 fs_manager.cc:359] Metadata directory not provided
I20260807 18:35:54.664656 26743 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:35:54.668830 26743 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.003s
I20260807 18:35:54.671173 26762 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:35:54.672823 26743 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 18:35:54.672971 26743 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/minicluster-data/master-0/data,/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/minicluster-data/master-0/wal
uuid: "c582c50c1bb949ab9b83a517755fc867"
format_stamp: "Formatted at 2026-08-07 18:35:53 on dist-test-slave-6k22"
server_key: "a6c139fcb30d72205b96015e416b6a55"
server_key_iv: "2e9c60b703742dee832cf5a9347181a9"
server_key_version: "encryptionkey@0"
I20260807 18:35:54.673607 26743 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:35:54.689311 26743 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:35:54.692721 26743 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:35:54.692965 26743 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:35:54.700874 26743 rpc_server.cc:307] RPC server started. Bound to: 127.11.14.126:41615
I20260807 18:35:54.701207 26821 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.14.126:41615 every 8 connection(s)
I20260807 18:35:54.702095 26743 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/minicluster-data/master-0/data/info.pb
I20260807 18:35:54.706864 26822 sys_catalog.cc:263] Verifying existing consensus state
I20260807 18:35:54.708745 26822 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c582c50c1bb949ab9b83a517755fc867: Bootstrap starting.
I20260807 18:35:54.711122 11321 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkz56XN/build/debug/bin/kudu as pid 26743
I20260807 18:35:54.711675 11321 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkz56XN/build/debug/bin/kudu
/tmp/dist-test-taskkz56XN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.14.65:38891
--local_ip_for_outbound_sockets=127.11.14.65
--tserver_master_addrs=127.11.14.126:41615
--webserver_port=37565
--webserver_interface=127.11.14.65
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.65.keytab
--principal=kudu/127.11.14.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.14.84:39981
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 18:35:54.727525 26822 log.cc:824] T 00000000000000000000000000000000 P c582c50c1bb949ab9b83a517755fc867: Log is configured to *not* fsync() on all Append() calls
I20260807 18:35:54.736721 26822 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c582c50c1bb949ab9b83a517755fc867: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 18:35:54.737138 26822 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c582c50c1bb949ab9b83a517755fc867: Bootstrap complete.
I20260807 18:35:54.741420 26822 raft_consensus.cc:348] T 00000000000000000000000000000000 P c582c50c1bb949ab9b83a517755fc867 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c582c50c1bb949ab9b83a517755fc867" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 41615 } }
I20260807 18:35:54.741848 26822 raft_consensus.cc:729] T 00000000000000000000000000000000 P c582c50c1bb949ab9b83a517755fc867 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c582c50c1bb949ab9b83a517755fc867, State: Initialized, Role: FOLLOWER
I20260807 18:35:54.742416 26822 consensus_queue.cc:260] T 00000000000000000000000000000000 P c582c50c1bb949ab9b83a517755fc867 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c582c50c1bb949ab9b83a517755fc867" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 41615 } }
I20260807 18:35:54.742573 26822 raft_consensus.cc:388] T 00000000000000000000000000000000 P c582c50c1bb949ab9b83a517755fc867 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:35:54.742676 26822 raft_consensus.cc:482] T 00000000000000000000000000000000 P c582c50c1bb949ab9b83a517755fc867 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:35:54.742806 26822 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c582c50c1bb949ab9b83a517755fc867 [term 1 FOLLOWER]: Advancing to term 2
I20260807 18:35:54.744855 26822 raft_consensus.cc:504] T 00000000000000000000000000000000 P c582c50c1bb949ab9b83a517755fc867 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c582c50c1bb949ab9b83a517755fc867" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 41615 } }
I20260807 18:35:54.745218 26822 leader_election.cc:302] T 00000000000000000000000000000000 P c582c50c1bb949ab9b83a517755fc867 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c582c50c1bb949ab9b83a517755fc867; no voters:
I20260807 18:35:54.745648 26822 leader_election.cc:288] T 00000000000000000000000000000000 P c582c50c1bb949ab9b83a517755fc867 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260807 18:35:54.745739 26827 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c582c50c1bb949ab9b83a517755fc867 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 18:35:54.745952 26827 raft_consensus.cc:686] T 00000000000000000000000000000000 P c582c50c1bb949ab9b83a517755fc867 [term 2 LEADER]: Becoming Leader. State: Replica: c582c50c1bb949ab9b83a517755fc867, State: Running, Role: LEADER
I20260807 18:35:54.746330 26827 consensus_queue.cc:237] T 00000000000000000000000000000000 P c582c50c1bb949ab9b83a517755fc867 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c582c50c1bb949ab9b83a517755fc867" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 41615 } }
I20260807 18:35:54.746997 26822 sys_catalog.cc:564] T 00000000000000000000000000000000 P c582c50c1bb949ab9b83a517755fc867 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:35:54.747875 26828 sys_catalog.cc:455] T 00000000000000000000000000000000 P c582c50c1bb949ab9b83a517755fc867 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "c582c50c1bb949ab9b83a517755fc867" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c582c50c1bb949ab9b83a517755fc867" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 41615 } } }
I20260807 18:35:54.748241 26828 sys_catalog.cc:458] T 00000000000000000000000000000000 P c582c50c1bb949ab9b83a517755fc867 [sys.catalog]: This master's current role is: LEADER
I20260807 18:35:54.748142 26829 sys_catalog.cc:455] T 00000000000000000000000000000000 P c582c50c1bb949ab9b83a517755fc867 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c582c50c1bb949ab9b83a517755fc867. Latest consensus state: current_term: 2 leader_uuid: "c582c50c1bb949ab9b83a517755fc867" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c582c50c1bb949ab9b83a517755fc867" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 41615 } } }
I20260807 18:35:54.748591 26829 sys_catalog.cc:458] T 00000000000000000000000000000000 P c582c50c1bb949ab9b83a517755fc867 [sys.catalog]: This master's current role is: LEADER
I20260807 18:35:54.748914 26838 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:35:54.753911 26838 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:35:54.754788 26838 catalog_manager.cc:1259] Loaded cluster ID: 46cec966d5954dee8323a97f8fce0c46
I20260807 18:35:54.754854 26838 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:35:54.756332 26838 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260807 18:35:54.760856 26838 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260807 18:35:54.766599 26838 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:35:54.767354 26838 catalog_manager.cc:5879] T 00000000000000000000000000000000 P c582c50c1bb949ab9b83a517755fc867: Loaded TSK: 0
I20260807 18:35:54.768589 26838 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 18:35:54.837910 26824 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:35:54.838193 26824 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:35:54.838238 26824 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:35:54.842432 26824 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:35:54.842507 26824 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:35:54.842568 26824 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.14.65
I20260807 18:35:54.848335 26824 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.14.84:39981
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.65.keytab
--principal=kudu/127.11.14.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.14.65:38891
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.14.65
--webserver_port=37565
--tserver_master_addrs=127.11.14.126:41615
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26824
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.14.65
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:35:54.849689 26824 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:35:54.850759 26824 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:35:54.861985 26852 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:35:54.862063 26855 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:35:54.862002 26853 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:35:54.862412 26824 server_base.cc:1034] running on GCE node
I20260807 18:35:54.862787 26824 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:35:54.863479 26824 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:35:54.864692 26824 hybrid_clock.cc:648] HybridClock initialized: now 1786127754864648 us; error 47 us; skew 500 ppm
Aug 07 18:35:54 dist-test-slave-6k22 krb5kdc[26133](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127754, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:35:54.867957 26824 init.cc:377] Logged in from keytab as kudu/127.11.14.65@KRBTEST.COM (short username kudu)
I20260807 18:35:54.869081 26824 webserver.cc:460] Webserver started at http://127.11.14.65:37565/ using document root <none> and password file <none>
I20260807 18:35:54.869731 26824 fs_manager.cc:359] Metadata directory not provided
I20260807 18:35:54.869858 26824 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:35:54.874867 26824 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.001s
I20260807 18:35:54.877617 26862 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:35:54.879293 26824 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 18:35:54.879446 26824 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/minicluster-data/ts-0/data,/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/minicluster-data/ts-0/wal
uuid: "e015739a99844c14829e0b989348e2f1"
format_stamp: "Formatted at 2026-08-07 18:35:53 on dist-test-slave-6k22"
server_key: "f11acd2898004152d4748cf435a289b1"
server_key_iv: "803ddf15583642fd88ed99d9769544b7"
server_key_version: "encryptionkey@0"
I20260807 18:35:54.880162 26824 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:35:54.900564 26824 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:35:54.903996 26824 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:35:54.904214 26824 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:35:54.904846 26824 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:35:54.905972 26870 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260807 18:35:54.910454 26824 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260807 18:35:54.910529 26824 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.000s sys 0.000s
I20260807 18:35:54.910632 26824 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260807 18:35:54.912743 26824 ts_tablet_manager.cc:610] Registered 1 tablets
I20260807 18:35:54.912813 26824 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.001s sys 0.000s
I20260807 18:35:54.913278 26870 tablet_bootstrap.cc:492] T 474f2cd763444a55a47c905282cde92f P e015739a99844c14829e0b989348e2f1: Bootstrap starting.
I20260807 18:35:54.923832 26824 rpc_server.cc:307] RPC server started. Bound to: 127.11.14.65:38891
I20260807 18:35:54.923919 26989 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.14.65:38891 every 8 connection(s)
I20260807 18:35:54.924882 26824 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/minicluster-data/ts-0/data/info.pb
I20260807 18:35:54.930559 11321 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkz56XN/build/debug/bin/kudu as pid 26824
I20260807 18:35:54.931224 11321 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkz56XN/build/debug/bin/kudu
/tmp/dist-test-taskkz56XN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.14.66:39165
--local_ip_for_outbound_sockets=127.11.14.66
--tserver_master_addrs=127.11.14.126:41615
--webserver_port=42919
--webserver_interface=127.11.14.66
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.66.keytab
--principal=kudu/127.11.14.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.14.84:39981
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 18:35:54.934237 26870 log.cc:824] T 474f2cd763444a55a47c905282cde92f P e015739a99844c14829e0b989348e2f1: Log is configured to *not* fsync() on all Append() calls
Aug 07 18:35:54 dist-test-slave-6k22 krb5kdc[26133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127754, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.65@KRBTEST.COM for kudu/127.11.14.126@KRBTEST.COM
I20260807 18:35:54.943487 26870 tablet_bootstrap.cc:492] T 474f2cd763444a55a47c905282cde92f P e015739a99844c14829e0b989348e2f1: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 18:35:54.944038 26870 tablet_bootstrap.cc:492] T 474f2cd763444a55a47c905282cde92f P e015739a99844c14829e0b989348e2f1: Bootstrap complete.
I20260807 18:35:54.945664 26870 ts_tablet_manager.cc:1397] T 474f2cd763444a55a47c905282cde92f P e015739a99844c14829e0b989348e2f1: Time spent bootstrapping tablet: real 0.033s user 0.018s sys 0.012s
I20260807 18:35:54.947098 26992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:35:54.927548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:41615 (local address 127.11.14.65:32887)
0807 18:35:54.928363 (+ 815us) negotiation.cc:107] Waiting for socket to connect
0807 18:35:54.928421 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0807 18:35:54.929329 (+ 908us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:35:54.931195 (+ 1866us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:35:54.931204 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:35:54.931789 (+ 585us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:35:54.932519 (+ 730us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:35:54.932544 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:35:54.934489 (+ 1945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:35:54.934493 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:35:54.936101 (+ 1608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:35:54.936118 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:35:54.937174 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:35:54.937181 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:35:54.937450 (+ 269us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:35:54.938272 (+ 822us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:35:54.938301 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:35:54.940481 (+ 2180us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:35:54.943215 (+ 2734us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:35:54.943225 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:35:54.943242 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:35:54.943701 (+ 459us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:35:54.944181 (+ 480us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:35:54.944185 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:35:54.944187 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:35:54.944362 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:35:54.945184 (+ 822us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:35:54.945194 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:35:54.945500 (+ 306us) client_negotiation.cc:769] Sending connection context
0807 18:35:54.945711 (+ 211us) client_negotiation.cc:240] Negotiation successful
0807 18:35:54.946077 (+ 366us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":574,"thread_start_us":197,"threads_started":1}
I20260807 18:35:54.948127 26993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:35:54.927740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.65:32887 (local address 127.11.14.126:41615)
0807 18:35:54.928177 (+ 437us) server_negotiation.cc:206] Beginning negotiation
0807 18:35:54.928199 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0807 18:35:54.928594 (+ 395us) server_negotiation.cc:407] Connection header received
0807 18:35:54.929815 (+ 1221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:35:54.929832 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:35:54.930431 (+ 599us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:35:54.930863 (+ 432us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:35:54.932759 (+ 1896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:35:54.934278 (+ 1519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:35:54.936265 (+ 1987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:35:54.936995 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:35:54.937180 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:35:54.940763 (+ 3583us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:35:54.940770 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:35:54.940786 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:35:54.940832 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:35:54.943059 (+ 2227us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:35:54.943876 (+ 817us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:35:54.943883 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:35:54.943889 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:35:54.944054 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:35:54.944482 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:35:54.944488 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:35:54.944492 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:35:54.944996 (+ 504us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:35:54.945126 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0807 18:35:54.945957 (+ 831us) server_negotiation.cc:299] Negotiation successful
0807 18:35:54.946360 (+ 403us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":268,"thread_start_us":131,"threads_started":1}
I20260807 18:35:54.948762 26870 raft_consensus.cc:348] T 474f2cd763444a55a47c905282cde92f P e015739a99844c14829e0b989348e2f1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e4cd084ed8d4729827ac3a8507c961d" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 39165 } } peers { permanent_uuid: "e015739a99844c14829e0b989348e2f1" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 38891 } } peers { permanent_uuid: "3981583d3760440d863c0aa8b1bfb361" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 35171 } }
I20260807 18:35:54.949286 26870 raft_consensus.cc:729] T 474f2cd763444a55a47c905282cde92f P e015739a99844c14829e0b989348e2f1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e015739a99844c14829e0b989348e2f1, State: Initialized, Role: FOLLOWER
I20260807 18:35:54.949821 26870 consensus_queue.cc:260] T 474f2cd763444a55a47c905282cde92f P e015739a99844c14829e0b989348e2f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e4cd084ed8d4729827ac3a8507c961d" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 39165 } } peers { permanent_uuid: "e015739a99844c14829e0b989348e2f1" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 38891 } } peers { permanent_uuid: "3981583d3760440d863c0aa8b1bfb361" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 35171 } }
I20260807 18:35:54.950870 26870 ts_tablet_manager.cc:1428] T 474f2cd763444a55a47c905282cde92f P e015739a99844c14829e0b989348e2f1: Time spent starting tablet: real 0.005s user 0.006s sys 0.001s
I20260807 18:35:54.950892 26990 heartbeater.cc:347] Connected to a master server at 127.11.14.126:41615
I20260807 18:35:54.951139 26990 heartbeater.cc:464] Registering TS with master...
I20260807 18:35:54.951756 26990 heartbeater.cc:511] Master 127.11.14.126:41615 requested a full tablet report, sending...
I20260807 18:35:54.954336 26780 ts_manager.cc:194] Registered new tserver with Master: e015739a99844c14829e0b989348e2f1 (127.11.14.65:38891)
I20260807 18:35:54.957828 26780 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.14.65@KRBTEST.COM'} at 127.11.14.65:32887
I20260807 18:35:54.960508 26990 heartbeater.cc:503] Master 127.11.14.126:41615 was elected leader, sending a full tablet report...
Aug 07 18:35:54 dist-test-slave-6k22 krb5kdc[26133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127754, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.126@KRBTEST.COM for kudu/127.11.14.65@KRBTEST.COM
I20260807 18:35:54.967372 26997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:35:54.957151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.65:38891 (local address 127.0.0.1:39414)
0807 18:35:54.957912 (+ 761us) negotiation.cc:107] Waiting for socket to connect
0807 18:35:54.957957 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 18:35:54.958086 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:35:54.958679 (+ 593us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:35:54.958685 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:35:54.959068 (+ 383us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:35:54.959170 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:35:54.959176 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:35:54.960222 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:35:54.960229 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:35:54.961002 (+ 773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:35:54.961013 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:35:54.961583 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:35:54.961587 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:35:54.961709 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:35:54.962291 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:35:54.962310 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:35:54.963689 (+ 1379us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:35:54.965531 (+ 1842us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:35:54.965543 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:35:54.965549 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:35:54.965780 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:35:54.966103 (+ 323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:35:54.966107 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:35:54.966109 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:35:54.966200 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:35:54.966678 (+ 478us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:35:54.966685 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:35:54.966816 (+ 131us) client_negotiation.cc:769] Sending connection context
0807 18:35:54.966967 (+ 151us) client_negotiation.cc:240] Negotiation successful
0807 18:35:54.967184 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":586,"thread_start_us":105,"threads_started":1}
I20260807 18:35:54.967373 26998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:35:54.957344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39414 (local address 127.11.14.65:38891)
0807 18:35:54.957711 (+ 367us) server_negotiation.cc:206] Beginning negotiation
0807 18:35:54.957726 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 18:35:54.958067 (+ 341us) server_negotiation.cc:407] Connection header received
0807 18:35:54.958260 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:35:54.958266 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:35:54.958491 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:35:54.958635 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:35:54.959301 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:35:54.960098 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:35:54.961124 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:35:54.961456 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:35:54.961581 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:35:54.963841 (+ 2260us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:35:54.963849 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:35:54.963860 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:35:54.963881 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:35:54.965409 (+ 1528us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:35:54.965907 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:35:54.965920 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:35:54.965921 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:35:54.965998 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:35:54.966298 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:35:54.966301 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:35:54.966303 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:35:54.966553 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:35:54.966705 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0807 18:35:54.967049 (+ 344us) server_negotiation.cc:299] Negotiation successful
0807 18:35:54.967210 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":275,"thread_start_us":88,"threads_started":1}
I20260807 18:35:54.968509 26915 tablet_service.cc:1514] Processing DeleteTablet for tablet 474f2cd763444a55a47c905282cde92f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:35:54 UTC) from {username='kudu', principal='kudu/127.11.14.126@KRBTEST.COM'} at 127.0.0.1:39414
I20260807 18:35:54.969389 26999 tablet_replica.cc:330] stopping tablet replica
I20260807 18:35:54.969698 26999 raft_consensus.cc:2229] T 474f2cd763444a55a47c905282cde92f P e015739a99844c14829e0b989348e2f1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:35:54.969970 26999 raft_consensus.cc:2258] T 474f2cd763444a55a47c905282cde92f P e015739a99844c14829e0b989348e2f1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:35:54.970963 26999 ts_tablet_manager.cc:1905] T 474f2cd763444a55a47c905282cde92f P e015739a99844c14829e0b989348e2f1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:35:54.974638 26999 ts_tablet_manager.cc:1918] T 474f2cd763444a55a47c905282cde92f P e015739a99844c14829e0b989348e2f1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:35:54.974725 26999 log.cc:1196] T 474f2cd763444a55a47c905282cde92f P e015739a99844c14829e0b989348e2f1: Deleting WAL directory at /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/minicluster-data/ts-0/wal/wals/474f2cd763444a55a47c905282cde92f
I20260807 18:35:54.975005 26999 ts_tablet_manager.cc:1939] T 474f2cd763444a55a47c905282cde92f P e015739a99844c14829e0b989348e2f1: Deleting consensus metadata
I20260807 18:35:54.976871 26763 catalog_manager.cc:4841] TS e015739a99844c14829e0b989348e2f1 (127.11.14.65:38891): tablet 474f2cd763444a55a47c905282cde92f (table test-table [id=7673f3c8278b4969abde6b453856b13e]) successfully deleted
W20260807 18:35:55.059202 26994 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:35:55.059451 26994 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:35:55.059495 26994 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:35:55.063733 26994 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:35:55.063794 26994 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:35:55.063854 26994 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.14.66
I20260807 18:35:55.069145 26994 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.14.84:39981
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.66.keytab
--principal=kudu/127.11.14.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.14.66:39165
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.14.66
--webserver_port=42919
--tserver_master_addrs=127.11.14.126:41615
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26994
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.14.66
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:35:55.070247 26994 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:35:55.071238 26994 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:35:55.081205 27005 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:35:55.081221 27007 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:35:55.081259 27004 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:35:55.081395 26994 server_base.cc:1034] running on GCE node
I20260807 18:35:55.081784 26994 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:35:55.082510 26994 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:35:55.083731 26994 hybrid_clock.cc:648] HybridClock initialized: now 1786127755083713 us; error 30 us; skew 500 ppm
Aug 07 18:35:55 dist-test-slave-6k22 krb5kdc[26133](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127755, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:35:55.086587 26994 init.cc:377] Logged in from keytab as kudu/127.11.14.66@KRBTEST.COM (short username kudu)
I20260807 18:35:55.087636 26994 webserver.cc:460] Webserver started at http://127.11.14.66:42919/ using document root <none> and password file <none>
I20260807 18:35:55.088244 26994 fs_manager.cc:359] Metadata directory not provided
I20260807 18:35:55.088362 26994 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:35:55.092538 26994 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 18:35:55.094933 27016 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:35:55.096372 26994 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 18:35:55.096530 26994 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/minicluster-data/ts-1/data,/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/minicluster-data/ts-1/wal
uuid: "5e4cd084ed8d4729827ac3a8507c961d"
format_stamp: "Formatted at 2026-08-07 18:35:53 on dist-test-slave-6k22"
server_key: "51256bb2d4887aeb74361bf8f1991c3b"
server_key_iv: "ab932955dd02add51fe6556f779490e1"
server_key_version: "encryptionkey@0"
I20260807 18:35:55.097127 26994 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:35:55.126868 26994 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:35:55.130148 26994 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:35:55.130393 26994 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:35:55.131044 26994 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:35:55.131960 26994 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:35:55.132040 26994 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:35:55.132109 26994 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:35:55.132164 26994 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:35:55.141789 26994 rpc_server.cc:307] RPC server started. Bound to: 127.11.14.66:39165
I20260807 18:35:55.141817 27143 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.14.66:39165 every 8 connection(s)
I20260807 18:35:55.142854 26994 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/minicluster-data/ts-1/data/info.pb
I20260807 18:35:55.150709 11321 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkz56XN/build/debug/bin/kudu as pid 26994
I20260807 18:35:55.151198 11321 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkz56XN/build/debug/bin/kudu
/tmp/dist-test-taskkz56XN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.14.67:35171
--local_ip_for_outbound_sockets=127.11.14.67
--tserver_master_addrs=127.11.14.126:41615
--webserver_port=33485
--webserver_interface=127.11.14.67
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.67.keytab
--principal=kudu/127.11.14.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.14.84:39981
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 18:35:55 dist-test-slave-6k22 krb5kdc[26133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127755, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.66@KRBTEST.COM for kudu/127.11.14.126@KRBTEST.COM
I20260807 18:35:55.159004 26993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:35:55.145026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.66:52993 (local address 127.11.14.126:41615)
0807 18:35:55.145181 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0807 18:35:55.145184 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 18:35:55.146158 (+ 974us) server_negotiation.cc:407] Connection header received
0807 18:35:55.147662 (+ 1504us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:35:55.147665 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:35:55.147718 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:35:55.147913 (+ 195us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:35:55.149395 (+ 1482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:35:55.149969 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:35:55.151205 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:35:55.151617 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:35:55.151752 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:35:55.154621 (+ 2869us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:35:55.154624 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:35:55.154627 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:35:55.154652 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:35:55.156314 (+ 1662us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:35:55.157138 (+ 824us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:35:55.157143 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:35:55.157146 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:35:55.157244 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:35:55.157711 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:35:55.157713 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:35:55.157714 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:35:55.157902 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:35:55.157981 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0807 18:35:55.158754 (+ 773us) server_negotiation.cc:299] Negotiation successful
0807 18:35:55.158877 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260807 18:35:55.160635 27146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:35:55.145504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:41615 (local address 127.11.14.66:52993)
0807 18:35:55.145995 (+ 491us) negotiation.cc:107] Waiting for socket to connect
0807 18:35:55.146041 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0807 18:35:55.147404 (+ 1363us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:35:55.148141 (+ 737us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:35:55.148152 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:35:55.148545 (+ 393us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:35:55.149234 (+ 689us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:35:55.149252 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:35:55.150085 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:35:55.150087 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:35:55.151069 (+ 982us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:35:55.151078 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:35:55.151797 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:35:55.151801 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:35:55.151992 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:35:55.152637 (+ 645us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:35:55.152655 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:35:55.154392 (+ 1737us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:35:55.156451 (+ 2059us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:35:55.156461 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:35:55.156482 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:35:55.156991 (+ 509us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:35:55.157361 (+ 370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:35:55.157369 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:35:55.157373 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:35:55.157596 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:35:55.158028 (+ 432us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:35:55.158037 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:35:55.158453 (+ 416us) client_negotiation.cc:769] Sending connection context
0807 18:35:55.158723 (+ 270us) client_negotiation.cc:240] Negotiation successful
0807 18:35:55.159284 (+ 561us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":296,"thread_start_us":133,"threads_started":1}
I20260807 18:35:55.162093 27144 heartbeater.cc:347] Connected to a master server at 127.11.14.126:41615
I20260807 18:35:55.162360 27144 heartbeater.cc:464] Registering TS with master...
I20260807 18:35:55.162886 27144 heartbeater.cc:511] Master 127.11.14.126:41615 requested a full tablet report, sending...
I20260807 18:35:55.164108 26780 ts_manager.cc:194] Registered new tserver with Master: 5e4cd084ed8d4729827ac3a8507c961d (127.11.14.66:39165)
I20260807 18:35:55.164697 26780 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.14.66@KRBTEST.COM'} at 127.11.14.66:52993
W20260807 18:35:55.275457 27147 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:35:55.275869 27147 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:35:55.275960 27147 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:35:55.280567 27147 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:35:55.280669 27147 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:35:55.280761 27147 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.14.67
I20260807 18:35:55.286441 27147 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.14.84:39981
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.67.keytab
--principal=kudu/127.11.14.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.14.67:35171
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.14.67
--webserver_port=33485
--tserver_master_addrs=127.11.14.126:41615
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27147
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.14.67
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:35:55.287667 27147 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:35:55.288741 27147 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:35:55.298584 27152 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:35:55.298791 27147 server_base.cc:1034] running on GCE node
W20260807 18:35:55.298584 27155 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:35:55.298584 27153 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:35:55.299268 27147 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:35:55.299996 27147 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:35:55.301158 27147 hybrid_clock.cc:648] HybridClock initialized: now 1786127755301118 us; error 51 us; skew 500 ppm
Aug 07 18:35:55 dist-test-slave-6k22 krb5kdc[26133](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127755, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:35:55.304162 27147 init.cc:377] Logged in from keytab as kudu/127.11.14.67@KRBTEST.COM (short username kudu)
I20260807 18:35:55.305030 27147 webserver.cc:460] Webserver started at http://127.11.14.67:33485/ using document root <none> and password file <none>
I20260807 18:35:55.305626 27147 fs_manager.cc:359] Metadata directory not provided
I20260807 18:35:55.305742 27147 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:35:55.309692 27147 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 18:35:55.312072 27163 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:35:55.313472 27147 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:35:55.313625 27147 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/minicluster-data/ts-2/data,/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/minicluster-data/ts-2/wal
uuid: "3981583d3760440d863c0aa8b1bfb361"
format_stamp: "Formatted at 2026-08-07 18:35:54 on dist-test-slave-6k22"
server_key: "14005ea34632b5c69f2de8014c948c38"
server_key_iv: "9eec48576c193ae9e9b9ec8ec50e0d22"
server_key_version: "encryptionkey@0"
I20260807 18:35:55.314214 27147 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:35:55.337271 27147 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:35:55.340741 27147 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:35:55.340970 27147 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:35:55.341630 27147 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:35:55.342528 27147 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:35:55.342608 27147 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:35:55.342682 27147 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:35:55.342741 27147 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:35:55.353960 27147 rpc_server.cc:307] RPC server started. Bound to: 127.11.14.67:35171
I20260807 18:35:55.354133 27287 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.14.67:35171 every 8 connection(s)
I20260807 18:35:55.355121 27147 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/minicluster-data/ts-2/data/info.pb
I20260807 18:35:55.360692 11321 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkz56XN/build/debug/bin/kudu as pid 27147
Aug 07 18:35:55 dist-test-slave-6k22 krb5kdc[26133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127755, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.67@KRBTEST.COM for kudu/127.11.14.126@KRBTEST.COM
I20260807 18:35:55.370097 27293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:35:55.361483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44800 (local address 127.11.14.126:41615)
0807 18:35:55.361811 (+ 328us) server_negotiation.cc:206] Beginning negotiation
0807 18:35:55.361814 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 18:35:55.361905 (+ 91us) server_negotiation.cc:407] Connection header received
0807 18:35:55.362052 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:35:55.362054 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:35:55.362102 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:35:55.362199 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:35:55.362926 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:35:55.363632 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:35:55.364481 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:35:55.364712 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:35:55.364838 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:35:55.366363 (+ 1525us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:35:55.366366 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:35:55.366368 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:35:55.366403 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:35:55.368329 (+ 1926us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:35:55.368819 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:35:55.368822 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:35:55.368824 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:35:55.368910 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:35:55.369222 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:35:55.369225 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:35:55.369226 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:35:55.369433 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:35:55.369658 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0807 18:35:55.369764 (+ 106us) server_negotiation.cc:299] Negotiation successful
0807 18:35:55.369891 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":263,"thread_start_us":102,"threads_started":1}
I20260807 18:35:55.371343 27291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:35:55.357658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:41615 (local address 127.11.14.67:35169)
0807 18:35:55.358130 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0807 18:35:55.358175 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 18:35:55.359362 (+ 1187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:35:55.359943 (+ 581us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:35:55.359951 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:35:55.360336 (+ 385us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:35:55.360890 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:35:55.360915 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:35:55.361915 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:35:55.361922 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:35:55.363099 (+ 1177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:35:55.363110 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:35:55.363661 (+ 551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:35:55.363669 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:35:55.363876 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:35:55.364507 (+ 631us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:35:55.364524 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:35:55.366003 (+ 1479us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:35:55.368008 (+ 2005us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:35:55.368015 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:35:55.368035 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:35:55.368466 (+ 431us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:35:55.368819 (+ 353us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:35:55.368822 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:35:55.368824 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:35:55.369017 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:35:55.369463 (+ 446us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:35:55.369473 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:35:55.369810 (+ 337us) client_negotiation.cc:769] Sending connection context
0807 18:35:55.370027 (+ 217us) client_negotiation.cc:240] Negotiation successful
0807 18:35:55.370401 (+ 374us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":118,"threads_started":1}
I20260807 18:35:55.372028 26993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:35:55.357246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.67:35169 (local address 127.11.14.126:41615)
0807 18:35:55.357430 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0807 18:35:55.357435 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:35:55.358299 (+ 864us) server_negotiation.cc:407] Connection header received
0807 18:35:55.359629 (+ 1330us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:35:55.359631 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:35:55.359682 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:35:55.359760 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:35:55.361059 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:35:55.361766 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:35:55.363226 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:35:55.363456 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:35:55.363537 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:35:55.366186 (+ 2649us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:35:55.366190 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:35:55.366192 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:35:55.366218 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:35:55.367881 (+ 1663us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:35:55.368601 (+ 720us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:35:55.368607 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:35:55.368611 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:35:55.368703 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:35:55.369116 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:35:55.369119 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:35:55.369121 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:35:55.369346 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:35:55.369424 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0807 18:35:55.371779 (+ 2355us) server_negotiation.cc:299] Negotiation successful
0807 18:35:55.371899 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":77}
I20260807 18:35:55.372864 27288 heartbeater.cc:347] Connected to a master server at 127.11.14.126:41615
I20260807 18:35:55.373082 27288 heartbeater.cc:464] Registering TS with master...
I20260807 18:35:55.373541 27288 heartbeater.cc:511] Master 127.11.14.126:41615 requested a full tablet report, sending...
I20260807 18:35:55.374512 26780 ts_manager.cc:194] Registered new tserver with Master: 3981583d3760440d863c0aa8b1bfb361 (127.11.14.67:35171)
I20260807 18:35:55.375082 26780 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.14.67@KRBTEST.COM'} at 127.11.14.67:35169
I20260807 18:35:55.384438 11321 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 18:35:55.389472 26993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:35:55.385166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44814 (local address 127.11.14.126:41615)
0807 18:35:55.385368 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0807 18:35:55.385374 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:35:55.385549 (+ 175us) server_negotiation.cc:407] Connection header received
0807 18:35:55.385760 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:35:55.385763 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:35:55.385886 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:35:55.386016 (+ 130us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:35:55.386293 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:35:55.386901 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:35:55.387898 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:35:55.388125 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:35:55.388250 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:35:55.388608 (+ 358us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:35:55.389069 (+ 461us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:35:55.389223 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0807 18:35:55.389265 (+ 42us) server_negotiation.cc:299] Negotiation successful
0807 18:35:55.389318 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105}
I20260807 18:35:55.390173 26780 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:44814:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 18:35:55.393642 26780 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 18:35:55.401886 26915 tablet_service.cc:1467] Processing CreateTablet for tablet 94ddb3a94032496fbbf9e18b2bcf0ddf (DEFAULT_TABLE table=test-table [id=721595406b8b4a4e86ae52a518e702e3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:35:55.402624 26915 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 94ddb3a94032496fbbf9e18b2bcf0ddf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:35:55.405757 26870 tablet_bootstrap.cc:492] T 94ddb3a94032496fbbf9e18b2bcf0ddf P e015739a99844c14829e0b989348e2f1: Bootstrap starting.
I20260807 18:35:55.406390 26997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:35:55.400686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.67:35171 (local address 127.0.0.1:54646)
0807 18:35:55.400828 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0807 18:35:55.400847 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 18:35:55.400937 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:35:55.401989 (+ 1052us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:35:55.401993 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:35:55.402013 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:35:55.402134 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:35:55.402140 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:35:55.403751 (+ 1611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:35:55.403755 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:35:55.404983 (+ 1228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:35:55.404995 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:35:55.405910 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:35:55.405915 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:35:55.406016 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:35:55.406074 (+ 58us) client_negotiation.cc:769] Sending connection context
0807 18:35:55.406163 (+ 89us) client_negotiation.cc:240] Negotiation successful
0807 18:35:55.406247 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53,"mutex_wait_us":68}
I20260807 18:35:55.406471 27299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:35:55.400993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54646 (local address 127.11.14.67:35171)
0807 18:35:55.401409 (+ 416us) server_negotiation.cc:206] Beginning negotiation
0807 18:35:55.401422 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 18:35:55.401446 (+ 24us) server_negotiation.cc:407] Connection header received
0807 18:35:55.401537 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:35:55.401544 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:35:55.401818 (+ 274us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:35:55.401990 (+ 172us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:35:55.402284 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:35:55.403597 (+ 1313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:35:55.405097 (+ 1500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:35:55.405759 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:35:55.405887 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:35:55.406054 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0807 18:35:55.406221 (+ 167us) server_negotiation.cc:299] Negotiation successful
0807 18:35:55.406327 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":293,"thread_start_us":201,"threads_started":1}
I20260807 18:35:55.406744 26870 tablet_bootstrap.cc:654] T 94ddb3a94032496fbbf9e18b2bcf0ddf P e015739a99844c14829e0b989348e2f1: Neither blocks nor log segments found. Creating new log.
I20260807 18:35:55.407582 27300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:35:55.401944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38520 (local address 127.11.14.66:39165)
0807 18:35:55.402313 (+ 369us) server_negotiation.cc:206] Beginning negotiation
0807 18:35:55.402340 (+ 27us) server_negotiation.cc:399] Waiting for connection header
0807 18:35:55.402362 (+ 22us) server_negotiation.cc:407] Connection header received
0807 18:35:55.402437 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:35:55.402448 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:35:55.402593 (+ 145us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:35:55.402767 (+ 174us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:35:55.402975 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:35:55.404407 (+ 1432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:35:55.405984 (+ 1577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:35:55.406804 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:35:55.406944 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:35:55.407091 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0807 18:35:55.407320 (+ 229us) server_negotiation.cc:299] Negotiation successful
0807 18:35:55.407427 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":265,"thread_start_us":162,"threads_started":1}
I20260807 18:35:55.408640 26870 tablet_bootstrap.cc:492] T 94ddb3a94032496fbbf9e18b2bcf0ddf P e015739a99844c14829e0b989348e2f1: No bootstrap required, opened a new log
I20260807 18:35:55.408737 26870 ts_tablet_manager.cc:1397] T 94ddb3a94032496fbbf9e18b2bcf0ddf P e015739a99844c14829e0b989348e2f1: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20260807 18:35:55.409192 26870 raft_consensus.cc:348] T 94ddb3a94032496fbbf9e18b2bcf0ddf P e015739a99844c14829e0b989348e2f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3981583d3760440d863c0aa8b1bfb361" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 35171 } } peers { permanent_uuid: "5e4cd084ed8d4729827ac3a8507c961d" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 39165 } } peers { permanent_uuid: "e015739a99844c14829e0b989348e2f1" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 38891 } }
I20260807 18:35:55.409327 26870 raft_consensus.cc:374] T 94ddb3a94032496fbbf9e18b2bcf0ddf P e015739a99844c14829e0b989348e2f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:35:55.409368 26870 raft_consensus.cc:729] T 94ddb3a94032496fbbf9e18b2bcf0ddf P e015739a99844c14829e0b989348e2f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e015739a99844c14829e0b989348e2f1, State: Initialized, Role: FOLLOWER
I20260807 18:35:55.409504 26870 consensus_queue.cc:260] T 94ddb3a94032496fbbf9e18b2bcf0ddf P e015739a99844c14829e0b989348e2f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3981583d3760440d863c0aa8b1bfb361" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 35171 } } peers { permanent_uuid: "5e4cd084ed8d4729827ac3a8507c961d" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 39165 } } peers { permanent_uuid: "e015739a99844c14829e0b989348e2f1" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 38891 } }
I20260807 18:35:55.409776 26870 ts_tablet_manager.cc:1428] T 94ddb3a94032496fbbf9e18b2bcf0ddf P e015739a99844c14829e0b989348e2f1: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260807 18:35:55.410053 27217 tablet_service.cc:1467] Processing CreateTablet for tablet 94ddb3a94032496fbbf9e18b2bcf0ddf (DEFAULT_TABLE table=test-table [id=721595406b8b4a4e86ae52a518e702e3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:35:55.411007 27217 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 94ddb3a94032496fbbf9e18b2bcf0ddf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:35:55.407363 27298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:35:55.401033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.66:39165 (local address 127.0.0.1:38520)
0807 18:35:55.401605 (+ 572us) negotiation.cc:107] Waiting for socket to connect
0807 18:35:55.401619 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 18:35:55.401725 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:35:55.402739 (+ 1014us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:35:55.402742 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:35:55.402761 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:35:55.402874 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:35:55.402880 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:35:55.404574 (+ 1694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:35:55.404579 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:35:55.405844 (+ 1265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:35:55.405858 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:35:55.406946 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:35:55.406949 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:35:55.407050 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:35:55.407076 (+ 26us) client_negotiation.cc:769] Sending connection context
0807 18:35:55.407153 (+ 77us) client_negotiation.cc:240] Negotiation successful
0807 18:35:55.407222 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":511,"mutex_wait_us":32,"thread_start_us":165,"threads_started":1}
I20260807 18:35:55.415688 27072 tablet_service.cc:1467] Processing CreateTablet for tablet 94ddb3a94032496fbbf9e18b2bcf0ddf (DEFAULT_TABLE table=test-table [id=721595406b8b4a4e86ae52a518e702e3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:35:55.416860 27072 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 94ddb3a94032496fbbf9e18b2bcf0ddf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:35:55.419729 27302 tablet_bootstrap.cc:492] T 94ddb3a94032496fbbf9e18b2bcf0ddf P 3981583d3760440d863c0aa8b1bfb361: Bootstrap starting.
I20260807 18:35:55.422137 27302 tablet_bootstrap.cc:654] T 94ddb3a94032496fbbf9e18b2bcf0ddf P 3981583d3760440d863c0aa8b1bfb361: Neither blocks nor log segments found. Creating new log.
I20260807 18:35:55.423208 27302 log.cc:824] T 94ddb3a94032496fbbf9e18b2bcf0ddf P 3981583d3760440d863c0aa8b1bfb361: Log is configured to *not* fsync() on all Append() calls
I20260807 18:35:55.424571 27304 tablet_bootstrap.cc:492] T 94ddb3a94032496fbbf9e18b2bcf0ddf P 5e4cd084ed8d4729827ac3a8507c961d: Bootstrap starting.
I20260807 18:35:55.425763 27302 tablet_bootstrap.cc:492] T 94ddb3a94032496fbbf9e18b2bcf0ddf P 3981583d3760440d863c0aa8b1bfb361: No bootstrap required, opened a new log
I20260807 18:35:55.425988 27302 ts_tablet_manager.cc:1397] T 94ddb3a94032496fbbf9e18b2bcf0ddf P 3981583d3760440d863c0aa8b1bfb361: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.002s
W20260807 18:35:55.426229 26991 tablet.cc:2369] T 94ddb3a94032496fbbf9e18b2bcf0ddf P e015739a99844c14829e0b989348e2f1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:35:55.427377 27304 tablet_bootstrap.cc:654] T 94ddb3a94032496fbbf9e18b2bcf0ddf P 5e4cd084ed8d4729827ac3a8507c961d: Neither blocks nor log segments found. Creating new log.
I20260807 18:35:55.428184 27304 log.cc:824] T 94ddb3a94032496fbbf9e18b2bcf0ddf P 5e4cd084ed8d4729827ac3a8507c961d: Log is configured to *not* fsync() on all Append() calls
I20260807 18:35:55.429414 27302 raft_consensus.cc:348] T 94ddb3a94032496fbbf9e18b2bcf0ddf P 3981583d3760440d863c0aa8b1bfb361 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3981583d3760440d863c0aa8b1bfb361" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 35171 } } peers { permanent_uuid: "5e4cd084ed8d4729827ac3a8507c961d" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 39165 } } peers { permanent_uuid: "e015739a99844c14829e0b989348e2f1" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 38891 } }
I20260807 18:35:55.429667 27302 raft_consensus.cc:374] T 94ddb3a94032496fbbf9e18b2bcf0ddf P 3981583d3760440d863c0aa8b1bfb361 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:35:55.429764 27302 raft_consensus.cc:729] T 94ddb3a94032496fbbf9e18b2bcf0ddf P 3981583d3760440d863c0aa8b1bfb361 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3981583d3760440d863c0aa8b1bfb361, State: Initialized, Role: FOLLOWER
I20260807 18:35:55.430203 27304 tablet_bootstrap.cc:492] T 94ddb3a94032496fbbf9e18b2bcf0ddf P 5e4cd084ed8d4729827ac3a8507c961d: No bootstrap required, opened a new log
I20260807 18:35:55.430270 27302 consensus_queue.cc:260] T 94ddb3a94032496fbbf9e18b2bcf0ddf P 3981583d3760440d863c0aa8b1bfb361 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3981583d3760440d863c0aa8b1bfb361" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 35171 } } peers { permanent_uuid: "5e4cd084ed8d4729827ac3a8507c961d" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 39165 } } peers { permanent_uuid: "e015739a99844c14829e0b989348e2f1" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 38891 } }
I20260807 18:35:55.430406 27304 ts_tablet_manager.cc:1397] T 94ddb3a94032496fbbf9e18b2bcf0ddf P 5e4cd084ed8d4729827ac3a8507c961d: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.003s
I20260807 18:35:55.431190 27288 heartbeater.cc:503] Master 127.11.14.126:41615 was elected leader, sending a full tablet report...
I20260807 18:35:55.431428 27302 ts_tablet_manager.cc:1428] T 94ddb3a94032496fbbf9e18b2bcf0ddf P 3981583d3760440d863c0aa8b1bfb361: Time spent starting tablet: real 0.005s user 0.004s sys 0.001s
I20260807 18:35:55.434243 27304 raft_consensus.cc:348] T 94ddb3a94032496fbbf9e18b2bcf0ddf P 5e4cd084ed8d4729827ac3a8507c961d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3981583d3760440d863c0aa8b1bfb361" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 35171 } } peers { permanent_uuid: "5e4cd084ed8d4729827ac3a8507c961d" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 39165 } } peers { permanent_uuid: "e015739a99844c14829e0b989348e2f1" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 38891 } }
I20260807 18:35:55.434482 27304 raft_consensus.cc:374] T 94ddb3a94032496fbbf9e18b2bcf0ddf P 5e4cd084ed8d4729827ac3a8507c961d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:35:55.434563 27304 raft_consensus.cc:729] T 94ddb3a94032496fbbf9e18b2bcf0ddf P 5e4cd084ed8d4729827ac3a8507c961d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e4cd084ed8d4729827ac3a8507c961d, State: Initialized, Role: FOLLOWER
I20260807 18:35:55.435024 27304 consensus_queue.cc:260] T 94ddb3a94032496fbbf9e18b2bcf0ddf P 5e4cd084ed8d4729827ac3a8507c961d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3981583d3760440d863c0aa8b1bfb361" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 35171 } } peers { permanent_uuid: "5e4cd084ed8d4729827ac3a8507c961d" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 39165 } } peers { permanent_uuid: "e015739a99844c14829e0b989348e2f1" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 38891 } }
I20260807 18:35:55.435710 27144 heartbeater.cc:503] Master 127.11.14.126:41615 was elected leader, sending a full tablet report...
I20260807 18:35:55.435962 27304 ts_tablet_manager.cc:1428] T 94ddb3a94032496fbbf9e18b2bcf0ddf P 5e4cd084ed8d4729827ac3a8507c961d: Time spent starting tablet: real 0.005s user 0.005s sys 0.002s
I20260807 18:35:55.512750 27306 raft_consensus.cc:482] T 94ddb3a94032496fbbf9e18b2bcf0ddf P 3981583d3760440d863c0aa8b1bfb361 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:35:55.513046 27306 raft_consensus.cc:504] T 94ddb3a94032496fbbf9e18b2bcf0ddf P 3981583d3760440d863c0aa8b1bfb361 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3981583d3760440d863c0aa8b1bfb361" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 35171 } } peers { permanent_uuid: "5e4cd084ed8d4729827ac3a8507c961d" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 39165 } } peers { permanent_uuid: "e015739a99844c14829e0b989348e2f1" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 38891 } }
I20260807 18:35:55.514391 27306 leader_election.cc:288] T 94ddb3a94032496fbbf9e18b2bcf0ddf P 3981583d3760440d863c0aa8b1bfb361 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e4cd084ed8d4729827ac3a8507c961d (127.11.14.66:39165), e015739a99844c14829e0b989348e2f1 (127.11.14.65:38891)
I20260807 18:35:55.519172 27291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:35:55.514597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.66:39165 (local address 127.11.14.67:54709)
0807 18:35:55.514809 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0807 18:35:55.514826 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 18:35:55.514927 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:35:55.515206 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:35:55.515210 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:35:55.515229 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:35:55.515378 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:35:55.515385 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:35:55.516406 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:35:55.516413 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:35:55.517565 (+ 1152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:35:55.517576 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:35:55.518702 (+ 1126us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:35:55.518706 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:35:55.518818 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:35:55.518852 (+ 34us) client_negotiation.cc:769] Sending connection context
0807 18:35:55.518931 (+ 79us) client_negotiation.cc:240] Negotiation successful
0807 18:35:55.519010 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129}
I20260807 18:35:55.519675 27308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:35:55.514678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.65:38891 (local address 127.11.14.67:54507)
0807 18:35:55.515031 (+ 353us) negotiation.cc:107] Waiting for socket to connect
0807 18:35:55.515047 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 18:35:55.515145 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:35:55.515706 (+ 561us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:35:55.515709 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:35:55.515726 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:35:55.515854 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:35:55.515865 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:35:55.517098 (+ 1233us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:35:55.517101 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:35:55.518127 (+ 1026us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:35:55.518142 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:35:55.519122 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:35:55.519127 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:35:55.519268 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:35:55.519307 (+ 39us) client_negotiation.cc:769] Sending connection context
0807 18:35:55.519415 (+ 108us) client_negotiation.cc:240] Negotiation successful
0807 18:35:55.519511 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":278,"thread_start_us":135,"threads_started":1}
I20260807 18:35:55.519717 27309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:35:55.515145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.67:54507 (local address 127.11.14.65:38891)
0807 18:35:55.515422 (+ 277us) server_negotiation.cc:206] Beginning negotiation
0807 18:35:55.515429 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:35:55.515445 (+ 16us) server_negotiation.cc:407] Connection header received
0807 18:35:55.515499 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:35:55.515501 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:35:55.515588 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:35:55.515698 (+ 110us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:35:55.516009 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:35:55.516927 (+ 918us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:35:55.518259 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:35:55.518987 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:35:55.519099 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:35:55.519259 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0807 18:35:55.519453 (+ 194us) server_negotiation.cc:299] Negotiation successful
0807 18:35:55.519552 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":194,"thread_start_us":103,"threads_started":1}
I20260807 18:35:55.519932 27300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:35:55.514788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.67:54709 (local address 127.11.14.66:39165)
0807 18:35:55.514912 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0807 18:35:55.514916 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:35:55.514933 (+ 17us) server_negotiation.cc:407] Connection header received
0807 18:35:55.515033 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:35:55.515036 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:35:55.515092 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:35:55.515185 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:35:55.515505 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:35:55.516301 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:35:55.517743 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:35:55.518573 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:35:55.518710 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:35:55.518796 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0807 18:35:55.519721 (+ 925us) server_negotiation.cc:299] Negotiation successful
0807 18:35:55.519813 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260807 18:35:55.520358 26938 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "94ddb3a94032496fbbf9e18b2bcf0ddf" candidate_uuid: "3981583d3760440d863c0aa8b1bfb361" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e015739a99844c14829e0b989348e2f1" is_pre_election: true
I20260807 18:35:55.520530 27094 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "94ddb3a94032496fbbf9e18b2bcf0ddf" candidate_uuid: "3981583d3760440d863c0aa8b1bfb361" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e4cd084ed8d4729827ac3a8507c961d" is_pre_election: true
I20260807 18:35:55.520720 26938 raft_consensus.cc:2454] T 94ddb3a94032496fbbf9e18b2bcf0ddf P e015739a99844c14829e0b989348e2f1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3981583d3760440d863c0aa8b1bfb361 in term 0.
I20260807 18:35:55.520910 27094 raft_consensus.cc:2454] T 94ddb3a94032496fbbf9e18b2bcf0ddf P 5e4cd084ed8d4729827ac3a8507c961d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3981583d3760440d863c0aa8b1bfb361 in term 0.
I20260807 18:35:55.521251 27165 leader_election.cc:302] T 94ddb3a94032496fbbf9e18b2bcf0ddf P 3981583d3760440d863c0aa8b1bfb361 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3981583d3760440d863c0aa8b1bfb361, e015739a99844c14829e0b989348e2f1; no voters:
I20260807 18:35:55.521618 27306 raft_consensus.cc:2781] T 94ddb3a94032496fbbf9e18b2bcf0ddf P 3981583d3760440d863c0aa8b1bfb361 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:35:55.521713 27306 raft_consensus.cc:482] T 94ddb3a94032496fbbf9e18b2bcf0ddf P 3981583d3760440d863c0aa8b1bfb361 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:35:55.521759 27306 raft_consensus.cc:3037] T 94ddb3a94032496fbbf9e18b2bcf0ddf P 3981583d3760440d863c0aa8b1bfb361 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:35:55.522833 27306 raft_consensus.cc:504] T 94ddb3a94032496fbbf9e18b2bcf0ddf P 3981583d3760440d863c0aa8b1bfb361 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3981583d3760440d863c0aa8b1bfb361" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 35171 } } peers { permanent_uuid: "5e4cd084ed8d4729827ac3a8507c961d" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 39165 } } peers { permanent_uuid: "e015739a99844c14829e0b989348e2f1" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 38891 } }
I20260807 18:35:55.523190 27306 leader_election.cc:288] T 94ddb3a94032496fbbf9e18b2bcf0ddf P 3981583d3760440d863c0aa8b1bfb361 [CANDIDATE]: Term 1 election: Requested vote from peers 5e4cd084ed8d4729827ac3a8507c961d (127.11.14.66:39165), e015739a99844c14829e0b989348e2f1 (127.11.14.65:38891)
I20260807 18:35:55.523519 27094 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "94ddb3a94032496fbbf9e18b2bcf0ddf" candidate_uuid: "3981583d3760440d863c0aa8b1bfb361" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e4cd084ed8d4729827ac3a8507c961d"
I20260807 18:35:55.523599 26938 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "94ddb3a94032496fbbf9e18b2bcf0ddf" candidate_uuid: "3981583d3760440d863c0aa8b1bfb361" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e015739a99844c14829e0b989348e2f1"
I20260807 18:35:55.523720 27094 raft_consensus.cc:3037] T 94ddb3a94032496fbbf9e18b2bcf0ddf P 5e4cd084ed8d4729827ac3a8507c961d [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:35:55.523743 26938 raft_consensus.cc:3037] T 94ddb3a94032496fbbf9e18b2bcf0ddf P e015739a99844c14829e0b989348e2f1 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:35:55.524803 26938 raft_consensus.cc:2454] T 94ddb3a94032496fbbf9e18b2bcf0ddf P e015739a99844c14829e0b989348e2f1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3981583d3760440d863c0aa8b1bfb361 in term 1.
I20260807 18:35:55.524933 27094 raft_consensus.cc:2454] T 94ddb3a94032496fbbf9e18b2bcf0ddf P 5e4cd084ed8d4729827ac3a8507c961d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3981583d3760440d863c0aa8b1bfb361 in term 1.
I20260807 18:35:55.525115 27165 leader_election.cc:302] T 94ddb3a94032496fbbf9e18b2bcf0ddf P 3981583d3760440d863c0aa8b1bfb361 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3981583d3760440d863c0aa8b1bfb361, e015739a99844c14829e0b989348e2f1; no voters:
I20260807 18:35:55.525295 27306 raft_consensus.cc:2781] T 94ddb3a94032496fbbf9e18b2bcf0ddf P 3981583d3760440d863c0aa8b1bfb361 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:35:55.525504 27306 raft_consensus.cc:686] T 94ddb3a94032496fbbf9e18b2bcf0ddf P 3981583d3760440d863c0aa8b1bfb361 [term 1 LEADER]: Becoming Leader. State: Replica: 3981583d3760440d863c0aa8b1bfb361, State: Running, Role: LEADER
I20260807 18:35:55.525874 27306 consensus_queue.cc:237] T 94ddb3a94032496fbbf9e18b2bcf0ddf P 3981583d3760440d863c0aa8b1bfb361 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3981583d3760440d863c0aa8b1bfb361" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 35171 } } peers { permanent_uuid: "5e4cd084ed8d4729827ac3a8507c961d" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 39165 } } peers { permanent_uuid: "e015739a99844c14829e0b989348e2f1" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 38891 } }
I20260807 18:35:55.529899 26779 catalog_manager.cc:5495] T 94ddb3a94032496fbbf9e18b2bcf0ddf P 3981583d3760440d863c0aa8b1bfb361 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3981583d3760440d863c0aa8b1bfb361 (127.11.14.67). New cstate: current_term: 1 leader_uuid: "3981583d3760440d863c0aa8b1bfb361" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3981583d3760440d863c0aa8b1bfb361" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 35171 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e4cd084ed8d4729827ac3a8507c961d" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 39165 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e015739a99844c14829e0b989348e2f1" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 38891 } health_report { overall_health: UNKNOWN } } }
I20260807 18:35:55.568979 27299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:35:55.565360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54652 (local address 127.11.14.67:35171)
0807 18:35:55.565509 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0807 18:35:55.565513 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:35:55.565528 (+ 15us) server_negotiation.cc:407] Connection header received
0807 18:35:55.565635 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:35:55.565638 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:35:55.565699 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:35:55.565791 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:35:55.566124 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:35:55.566622 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:35:55.567442 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:35:55.567668 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:35:55.567745 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:35:55.568024 (+ 279us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:35:55.568463 (+ 439us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:35:55.568557 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0807 18:35:55.568671 (+ 114us) server_negotiation.cc:299] Negotiation successful
0807 18:35:55.568726 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260807 18:35:55.575230 26938 raft_consensus.cc:1261] T 94ddb3a94032496fbbf9e18b2bcf0ddf P e015739a99844c14829e0b989348e2f1 [term 1 FOLLOWER]: Refusing update from remote peer 3981583d3760440d863c0aa8b1bfb361: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:35:55.575235 27094 raft_consensus.cc:1261] T 94ddb3a94032496fbbf9e18b2bcf0ddf P 5e4cd084ed8d4729827ac3a8507c961d [term 1 FOLLOWER]: Refusing update from remote peer 3981583d3760440d863c0aa8b1bfb361: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:35:55.576025 27310 consensus_queue.cc:1035] T 94ddb3a94032496fbbf9e18b2bcf0ddf P 3981583d3760440d863c0aa8b1bfb361 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e4cd084ed8d4729827ac3a8507c961d" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 39165 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:35:55.576267 27306 consensus_queue.cc:1035] T 94ddb3a94032496fbbf9e18b2bcf0ddf P 3981583d3760440d863c0aa8b1bfb361 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e015739a99844c14829e0b989348e2f1" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 38891 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:35:55.584365 27315 mvcc.cc:205] Tried to move back new op lower bound from 7315979286828240896 to 7315979286637244416. Current Snapshot: MvccSnapshot[applied={T|T < 7315979286828240896}]
I20260807 18:35:55.593956 26779 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:44814:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:35:55.594128 26779 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:44814:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:35:55.596724 26779 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c582c50c1bb949ab9b83a517755fc867: Sending DeleteTablet for 3 replicas of tablet 94ddb3a94032496fbbf9e18b2bcf0ddf
I20260807 18:35:55.597592 11321 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkz56XN/build/debug/bin/kudu with pid 26824
I20260807 18:35:55.597544 27217 tablet_service.cc:1514] Processing DeleteTablet for tablet 94ddb3a94032496fbbf9e18b2bcf0ddf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:35:55 UTC) from {username='kudu', principal='kudu/127.11.14.126@KRBTEST.COM'} at 127.0.0.1:54646
I20260807 18:35:55.597610 27072 tablet_service.cc:1514] Processing DeleteTablet for tablet 94ddb3a94032496fbbf9e18b2bcf0ddf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:35:55 UTC) from {username='kudu', principal='kudu/127.11.14.126@KRBTEST.COM'} at 127.0.0.1:38520
I20260807 18:35:55.601778 27322 tablet_replica.cc:330] T 94ddb3a94032496fbbf9e18b2bcf0ddf P 5e4cd084ed8d4729827ac3a8507c961d: stopping tablet replica
I20260807 18:35:55.602097 27322 raft_consensus.cc:2229] T 94ddb3a94032496fbbf9e18b2bcf0ddf P 5e4cd084ed8d4729827ac3a8507c961d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:35:55.602392 27322 raft_consensus.cc:2258] T 94ddb3a94032496fbbf9e18b2bcf0ddf P 5e4cd084ed8d4729827ac3a8507c961d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:35:55.604180 27323 tablet_replica.cc:330] T 94ddb3a94032496fbbf9e18b2bcf0ddf P 3981583d3760440d863c0aa8b1bfb361: stopping tablet replica
I20260807 18:35:55.604570 27323 raft_consensus.cc:2229] T 94ddb3a94032496fbbf9e18b2bcf0ddf P 3981583d3760440d863c0aa8b1bfb361 [term 1 LEADER]: Raft consensus shutting down.
I20260807 18:35:55.604949 27322 ts_tablet_manager.cc:1905] T 94ddb3a94032496fbbf9e18b2bcf0ddf P 5e4cd084ed8d4729827ac3a8507c961d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:35:55.605086 27323 raft_consensus.cc:2258] T 94ddb3a94032496fbbf9e18b2bcf0ddf P 3981583d3760440d863c0aa8b1bfb361 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:35:55.606860 27323 ts_tablet_manager.cc:1905] T 94ddb3a94032496fbbf9e18b2bcf0ddf P 3981583d3760440d863c0aa8b1bfb361: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:35:55.608829 27322 ts_tablet_manager.cc:1918] T 94ddb3a94032496fbbf9e18b2bcf0ddf P 5e4cd084ed8d4729827ac3a8507c961d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:35:55.608971 27322 log.cc:1196] T 94ddb3a94032496fbbf9e18b2bcf0ddf P 5e4cd084ed8d4729827ac3a8507c961d: Deleting WAL directory at /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/minicluster-data/ts-1/wal/wals/94ddb3a94032496fbbf9e18b2bcf0ddf
I20260807 18:35:55.609328 27322 ts_tablet_manager.cc:1939] T 94ddb3a94032496fbbf9e18b2bcf0ddf P 5e4cd084ed8d4729827ac3a8507c961d: Deleting consensus metadata
I20260807 18:35:55.610066 27323 ts_tablet_manager.cc:1918] T 94ddb3a94032496fbbf9e18b2bcf0ddf P 3981583d3760440d863c0aa8b1bfb361: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:35:55.610209 27323 log.cc:1196] T 94ddb3a94032496fbbf9e18b2bcf0ddf P 3981583d3760440d863c0aa8b1bfb361: Deleting WAL directory at /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/minicluster-data/ts-2/wal/wals/94ddb3a94032496fbbf9e18b2bcf0ddf
I20260807 18:35:55.610642 27323 ts_tablet_manager.cc:1939] T 94ddb3a94032496fbbf9e18b2bcf0ddf P 3981583d3760440d863c0aa8b1bfb361: Deleting consensus metadata
I20260807 18:35:55.610800 26765 catalog_manager.cc:4841] TS 5e4cd084ed8d4729827ac3a8507c961d (127.11.14.66:39165): tablet 94ddb3a94032496fbbf9e18b2bcf0ddf (table test-table [id=721595406b8b4a4e86ae52a518e702e3]) successfully deleted
W20260807 18:35:55.611788 26763 connection.cc:535] client connection to 127.11.14.65:38891 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260807 18:35:55.611937 26763 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260807 18:35:55.612097 26766 catalog_manager.cc:4841] TS 3981583d3760440d863c0aa8b1bfb361 (127.11.14.67:35171): tablet 94ddb3a94032496fbbf9e18b2bcf0ddf (table test-table [id=721595406b8b4a4e86ae52a518e702e3]) successfully deleted
I20260807 18:35:55.612869 11321 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkz56XN/build/debug/bin/kudu with pid 26994
I20260807 18:35:55.613158 27298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:35:55.612843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.65:38891 (local address 127.0.0.1:39420)
0807 18:35:55.612987 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0807 18:35:55.613062 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.65:38891: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260807 18:35:55.615604 26763 catalog_manager.cc:4584] TS e015739a99844c14829e0b989348e2f1 (127.11.14.65:38891): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 94ddb3a94032496fbbf9e18b2bcf0ddf: Network error: Client connection negotiation failed: client connection to 127.11.14.65:38891: connect: Connection refused (error 111)
I20260807 18:35:55.627107 11321 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkz56XN/build/debug/bin/kudu with pid 27147
I20260807 18:35:55.634879 11321 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkz56XN/build/debug/bin/kudu with pid 26743
I20260807 18:35:55.643278 11321 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkz56XN/build/debug/bin/kudu
/tmp/dist-test-taskkz56XN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.14.126:41615
--webserver_interface=127.11.14.126
--webserver_port=44139
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.126.keytab
--principal=kudu/127.11.14.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.14.84:39981
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.14.126:41615
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:35:55.760424 27325 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:35:55.760751 27325 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:35:55.760834 27325 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:35:55.764882 27325 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:35:55.764989 27325 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:35:55.765028 27325 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:35:55.765062 27325 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:35:55.765093 27325 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:35:55.770406 27325 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.14.84:39981
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.14.126:41615
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.126.keytab
--principal=kudu/127.11.14.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.14.126:41615
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.14.126
--webserver_port=44139
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27325
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:35:55.771524 27325 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:35:55.772624 27325 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:35:55.781749 27334 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:35:55.781770 27332 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:35:55.781888 27325 server_base.cc:1034] running on GCE node
W20260807 18:35:55.781770 27331 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:35:55.782783 27325 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:35:55.784132 27325 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:35:55.785336 27325 hybrid_clock.cc:648] HybridClock initialized: now 1786127755785310 us; error 32 us; skew 500 ppm
Aug 07 18:35:55 dist-test-slave-6k22 krb5kdc[26133](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127755, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:35:55.788353 27325 init.cc:377] Logged in from keytab as kudu/127.11.14.126@KRBTEST.COM (short username kudu)
I20260807 18:35:55.789283 27325 webserver.cc:460] Webserver started at http://127.11.14.126:44139/ using document root <none> and password file <none>
I20260807 18:35:55.789894 27325 fs_manager.cc:359] Metadata directory not provided
I20260807 18:35:55.789997 27325 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:35:55.794270 27325 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 18:35:55.796708 27341 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:35:55.797982 27325 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:35:55.798136 27325 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/minicluster-data/master-0/data,/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/minicluster-data/master-0/wal
uuid: "c582c50c1bb949ab9b83a517755fc867"
format_stamp: "Formatted at 2026-08-07 18:35:53 on dist-test-slave-6k22"
server_key: "a6c139fcb30d72205b96015e416b6a55"
server_key_iv: "2e9c60b703742dee832cf5a9347181a9"
server_key_version: "encryptionkey@0"
I20260807 18:35:55.798729 27325 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:35:55.816561 27325 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:35:55.819773 27325 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:35:55.819995 27325 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:35:55.828042 27325 rpc_server.cc:307] RPC server started. Bound to: 127.11.14.126:41615
I20260807 18:35:55.828394 27399 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.14.126:41615 every 8 connection(s)
I20260807 18:35:55.829233 27325 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/minicluster-data/master-0/data/info.pb
I20260807 18:35:55.831471 11321 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkz56XN/build/debug/bin/kudu as pid 27325
I20260807 18:35:55.831961 11321 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkz56XN/build/debug/bin/kudu
/tmp/dist-test-taskkz56XN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.14.65:38891
--local_ip_for_outbound_sockets=127.11.14.65
--tserver_master_addrs=127.11.14.126:41615
--webserver_port=37565
--webserver_interface=127.11.14.65
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.65.keytab
--principal=kudu/127.11.14.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.14.84:39981
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 18:35:55.834524 27401 sys_catalog.cc:263] Verifying existing consensus state
I20260807 18:35:55.836479 27401 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c582c50c1bb949ab9b83a517755fc867: Bootstrap starting.
I20260807 18:35:55.855911 27401 log.cc:824] T 00000000000000000000000000000000 P c582c50c1bb949ab9b83a517755fc867: Log is configured to *not* fsync() on all Append() calls
I20260807 18:35:55.867360 27401 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c582c50c1bb949ab9b83a517755fc867: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 18:35:55.867844 27401 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c582c50c1bb949ab9b83a517755fc867: Bootstrap complete.
I20260807 18:35:55.872056 27401 raft_consensus.cc:348] T 00000000000000000000000000000000 P c582c50c1bb949ab9b83a517755fc867 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c582c50c1bb949ab9b83a517755fc867" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 41615 } }
I20260807 18:35:55.872481 27401 raft_consensus.cc:729] T 00000000000000000000000000000000 P c582c50c1bb949ab9b83a517755fc867 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: c582c50c1bb949ab9b83a517755fc867, State: Initialized, Role: FOLLOWER
I20260807 18:35:55.872987 27401 consensus_queue.cc:260] T 00000000000000000000000000000000 P c582c50c1bb949ab9b83a517755fc867 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c582c50c1bb949ab9b83a517755fc867" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 41615 } }
I20260807 18:35:55.873139 27401 raft_consensus.cc:388] T 00000000000000000000000000000000 P c582c50c1bb949ab9b83a517755fc867 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:35:55.873232 27401 raft_consensus.cc:482] T 00000000000000000000000000000000 P c582c50c1bb949ab9b83a517755fc867 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:35:55.873368 27401 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c582c50c1bb949ab9b83a517755fc867 [term 2 FOLLOWER]: Advancing to term 3
I20260807 18:35:55.875334 27401 raft_consensus.cc:504] T 00000000000000000000000000000000 P c582c50c1bb949ab9b83a517755fc867 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c582c50c1bb949ab9b83a517755fc867" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 41615 } }
I20260807 18:35:55.875753 27401 leader_election.cc:302] T 00000000000000000000000000000000 P c582c50c1bb949ab9b83a517755fc867 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c582c50c1bb949ab9b83a517755fc867; no voters:
I20260807 18:35:55.876129 27401 leader_election.cc:288] T 00000000000000000000000000000000 P c582c50c1bb949ab9b83a517755fc867 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260807 18:35:55.876255 27405 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c582c50c1bb949ab9b83a517755fc867 [term 3 FOLLOWER]: Leader election won for term 3
I20260807 18:35:55.876449 27405 raft_consensus.cc:686] T 00000000000000000000000000000000 P c582c50c1bb949ab9b83a517755fc867 [term 3 LEADER]: Becoming Leader. State: Replica: c582c50c1bb949ab9b83a517755fc867, State: Running, Role: LEADER
I20260807 18:35:55.876761 27405 consensus_queue.cc:237] T 00000000000000000000000000000000 P c582c50c1bb949ab9b83a517755fc867 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c582c50c1bb949ab9b83a517755fc867" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 41615 } }
I20260807 18:35:55.877545 27401 sys_catalog.cc:564] T 00000000000000000000000000000000 P c582c50c1bb949ab9b83a517755fc867 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:35:55.878223 27406 sys_catalog.cc:455] T 00000000000000000000000000000000 P c582c50c1bb949ab9b83a517755fc867 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "c582c50c1bb949ab9b83a517755fc867" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c582c50c1bb949ab9b83a517755fc867" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 41615 } } }
I20260807 18:35:55.878373 27406 sys_catalog.cc:458] T 00000000000000000000000000000000 P c582c50c1bb949ab9b83a517755fc867 [sys.catalog]: This master's current role is: LEADER
I20260807 18:35:55.878716 27408 sys_catalog.cc:455] T 00000000000000000000000000000000 P c582c50c1bb949ab9b83a517755fc867 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c582c50c1bb949ab9b83a517755fc867. Latest consensus state: current_term: 3 leader_uuid: "c582c50c1bb949ab9b83a517755fc867" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c582c50c1bb949ab9b83a517755fc867" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 41615 } } }
I20260807 18:35:55.878821 27408 sys_catalog.cc:458] T 00000000000000000000000000000000 P c582c50c1bb949ab9b83a517755fc867 [sys.catalog]: This master's current role is: LEADER
I20260807 18:35:55.879853 27412 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:35:55.885520 27412 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:35:55.886169 27412 catalog_manager.cc:1259] Loaded cluster ID: 46cec966d5954dee8323a97f8fce0c46
I20260807 18:35:55.886232 27412 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260807 18:35:55.887408 27412 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f9abcbc49ad google::LogMessage::Fail() at ??:0
@ 0x7f9abcbc788a google::LogMessage::SendToLog() at ??:0
@ 0x7f9abcbc44de google::LogMessage::Flush() at ??:0
@ 0x7f9abcbc5a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f9ac7503fdd _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7f9ac7504d2f kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f9ac7501579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7f9ac7545640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f9abee163d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f9abee16c83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7f9abee183f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f9abee07512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f9ac044b6db start_thread at ??:0
@ 0x7f9abb6c261f clone at ??:0
W20260807 18:35:55.989657 27403 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:35:55.989938 27403 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:35:55.989984 27403 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:35:55.994089 27403 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:35:55.994158 27403 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:35:55.994218 27403 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.14.65
I20260807 18:35:55.999727 27403 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.14.84:39981
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.65.keytab
--principal=kudu/127.11.14.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.14.65:38891
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.14.65
--webserver_port=37565
--tserver_master_addrs=127.11.14.126:41615
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27403
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.14.65
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:35:56.000902 27403 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:35:56.001981 27403 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:35:56.014132 27445 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:35:56.014277 27448 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:35:56.014422 27403 server_base.cc:1034] running on GCE node
W20260807 18:35:56.014215 27446 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:35:56.015308 27403 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:35:56.016062 27403 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:35:56.017259 27403 hybrid_clock.cc:648] HybridClock initialized: now 1786127756017211 us; error 71 us; skew 500 ppm
Aug 07 18:35:56 dist-test-slave-6k22 krb5kdc[26133](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127756, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:35:56.020785 27403 init.cc:377] Logged in from keytab as kudu/127.11.14.65@KRBTEST.COM (short username kudu)
I20260807 18:35:56.021806 27403 webserver.cc:460] Webserver started at http://127.11.14.65:37565/ using document root <none> and password file <none>
I20260807 18:35:56.022492 27403 fs_manager.cc:359] Metadata directory not provided
I20260807 18:35:56.022569 27403 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:35:56.026841 27403 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260807 18:35:56.029459 27457 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:35:56.030943 27403 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 18:35:56.031088 27403 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/minicluster-data/ts-0/data,/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/minicluster-data/ts-0/wal
uuid: "e015739a99844c14829e0b989348e2f1"
format_stamp: "Formatted at 2026-08-07 18:35:53 on dist-test-slave-6k22"
server_key: "f11acd2898004152d4748cf435a289b1"
server_key_iv: "803ddf15583642fd88ed99d9769544b7"
server_key_version: "encryptionkey@0"
I20260807 18:35:56.031752 27403 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:35:56.050112 27403 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:35:56.053725 27403 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:35:56.053954 27403 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:35:56.054598 27403 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:35:56.055815 27464 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260807 18:35:56.062160 27403 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260807 18:35:56.062256 27403 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.007s user 0.001s sys 0.000s
I20260807 18:35:56.062340 27403 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260807 18:35:56.064206 27403 ts_tablet_manager.cc:610] Registered 1 tablets
I20260807 18:35:56.064277 27403 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.001s sys 0.000s
I20260807 18:35:56.064571 27464 tablet_bootstrap.cc:492] T 94ddb3a94032496fbbf9e18b2bcf0ddf P e015739a99844c14829e0b989348e2f1: Bootstrap starting.
I20260807 18:35:56.075336 27403 rpc_server.cc:307] RPC server started. Bound to: 127.11.14.65:38891
I20260807 18:35:56.075546 27583 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.14.65:38891 every 8 connection(s)
I20260807 18:35:56.076432 27403 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/minicluster-data/ts-0/data/info.pb
I20260807 18:35:56.081028 11321 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkz56XN/build/debug/bin/kudu as pid 27403
I20260807 18:35:56.081467 11321 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkz56XN/build/debug/bin/kudu
/tmp/dist-test-taskkz56XN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.14.66:39165
--local_ip_for_outbound_sockets=127.11.14.66
--tserver_master_addrs=127.11.14.126:41615
--webserver_port=42919
--webserver_interface=127.11.14.66
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.66.keytab
--principal=kudu/127.11.14.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.14.84:39981
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 18:35:56.090750 27464 log.cc:824] T 94ddb3a94032496fbbf9e18b2bcf0ddf P e015739a99844c14829e0b989348e2f1: Log is configured to *not* fsync() on all Append() calls
I20260807 18:35:56.098137 27464 tablet_bootstrap.cc:492] T 94ddb3a94032496fbbf9e18b2bcf0ddf P e015739a99844c14829e0b989348e2f1: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 18:35:56.098559 27464 tablet_bootstrap.cc:492] T 94ddb3a94032496fbbf9e18b2bcf0ddf P e015739a99844c14829e0b989348e2f1: Bootstrap complete.
I20260807 18:35:56.099850 27464 ts_tablet_manager.cc:1397] T 94ddb3a94032496fbbf9e18b2bcf0ddf P e015739a99844c14829e0b989348e2f1: Time spent bootstrapping tablet: real 0.036s user 0.019s sys 0.013s
I20260807 18:35:56.102324 27464 raft_consensus.cc:348] T 94ddb3a94032496fbbf9e18b2bcf0ddf P e015739a99844c14829e0b989348e2f1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3981583d3760440d863c0aa8b1bfb361" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 35171 } } peers { permanent_uuid: "5e4cd084ed8d4729827ac3a8507c961d" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 39165 } } peers { permanent_uuid: "e015739a99844c14829e0b989348e2f1" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 38891 } }
I20260807 18:35:56.102777 27464 raft_consensus.cc:729] T 94ddb3a94032496fbbf9e18b2bcf0ddf P e015739a99844c14829e0b989348e2f1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e015739a99844c14829e0b989348e2f1, State: Initialized, Role: FOLLOWER
I20260807 18:35:56.103288 27464 consensus_queue.cc:260] T 94ddb3a94032496fbbf9e18b2bcf0ddf P e015739a99844c14829e0b989348e2f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3981583d3760440d863c0aa8b1bfb361" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 35171 } } peers { permanent_uuid: "5e4cd084ed8d4729827ac3a8507c961d" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 39165 } } peers { permanent_uuid: "e015739a99844c14829e0b989348e2f1" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 38891 } }
I20260807 18:35:56.104329 27464 ts_tablet_manager.cc:1428] T 94ddb3a94032496fbbf9e18b2bcf0ddf P e015739a99844c14829e0b989348e2f1: Time spent starting tablet: real 0.004s user 0.000s sys 0.004s
W20260807 18:35:56.181085 27586 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:35:56.078541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:41615 (local address 127.11.14.65:48083)
0807 18:35:56.079060 (+ 519us) negotiation.cc:107] Waiting for socket to connect
0807 18:35:56.079116 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0807 18:35:56.080029 (+ 913us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:35:56.180078 (+100049us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.126:41615: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":300,"thread_start_us":167,"threads_started":1}
W20260807 18:35:56.182509 27584 heartbeater.cc:644] Failed to heartbeat to 127.11.14.126:41615 (0 consecutive failures): Network error: Failed to ping master at 127.11.14.126:41615: Client connection negotiation failed: client connection to 127.11.14.126:41615: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260807 18:35:56.183375 27586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:35:56.183065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:41615 (local address 127.11.14.65:55615)
0807 18:35:56.183221 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0807 18:35:56.183304 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.126:41615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260807 18:35:56.184139 27586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:35:56.183896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:41615 (local address 127.11.14.65:49947)
0807 18:35:56.184017 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0807 18:35:56.184071 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.126:41615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260807 18:35:56.184372 27584 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 18:35:56.241410 27587 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:35:56.241679 27587 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:35:56.241724 27587 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:35:56.245882 27587 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:35:56.245942 27587 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:35:56.246057 27587 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.14.66
I20260807 18:35:56.251441 27587 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.14.84:39981
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.66.keytab
--principal=kudu/127.11.14.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.14.66:39165
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.14.66
--webserver_port=42919
--tserver_master_addrs=127.11.14.126:41615
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27587
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.14.66
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:35:56.252599 27587 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:35:56.253749 27587 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:35:56.263962 27599 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:35:56.264016 27587 server_base.cc:1034] running on GCE node
W20260807 18:35:56.263986 27597 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:35:56.264191 27595 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:35:56.264590 27587 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:35:56.265285 27587 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:35:56.266505 27587 hybrid_clock.cc:648] HybridClock initialized: now 1786127756266476 us; error 48 us; skew 500 ppm
Aug 07 18:35:56 dist-test-slave-6k22 krb5kdc[26133](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127756, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:35:56.269425 27587 init.cc:377] Logged in from keytab as kudu/127.11.14.66@KRBTEST.COM (short username kudu)
I20260807 18:35:56.270319 27587 webserver.cc:460] Webserver started at http://127.11.14.66:42919/ using document root <none> and password file <none>
I20260807 18:35:56.270869 27587 fs_manager.cc:359] Metadata directory not provided
I20260807 18:35:56.270931 27587 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:35:56.274994 27587 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260807 18:35:56.277311 27606 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:35:56.278736 27587 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 18:35:56.278843 27587 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/minicluster-data/ts-1/data,/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/minicluster-data/ts-1/wal
uuid: "5e4cd084ed8d4729827ac3a8507c961d"
format_stamp: "Formatted at 2026-08-07 18:35:53 on dist-test-slave-6k22"
server_key: "51256bb2d4887aeb74361bf8f1991c3b"
server_key_iv: "ab932955dd02add51fe6556f779490e1"
server_key_version: "encryptionkey@0"
I20260807 18:35:56.279429 27587 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:35:56.292388 27587 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:35:56.295933 27587 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:35:56.296094 27587 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:35:56.296650 27587 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:35:56.297458 27587 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:35:56.297508 27587 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:35:56.297547 27587 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:35:56.297562 27587 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:35:56.307245 27587 rpc_server.cc:307] RPC server started. Bound to: 127.11.14.66:39165
I20260807 18:35:56.307267 27731 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.14.66:39165 every 8 connection(s)
I20260807 18:35:56.308315 27587 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/minicluster-data/ts-1/data/info.pb
I20260807 18:35:56.312330 27734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:35:56.310502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:41615 (local address 127.11.14.66:40487)
0807 18:35:56.310978 (+ 476us) negotiation.cc:107] Waiting for socket to connect
0807 18:35:56.311093 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.126:41615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":119,"threads_started":1}
W20260807 18:35:56.312898 27732 heartbeater.cc:644] Failed to heartbeat to 127.11.14.126:41615 (0 consecutive failures): Network error: Failed to ping master at 127.11.14.126:41615: Client connection negotiation failed: client connection to 127.11.14.126:41615: connect: Connection refused (error 111)
I20260807 18:35:56.313508 27734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:35:56.313291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:41615 (local address 127.11.14.66:56777)
0807 18:35:56.313408 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0807 18:35:56.313454 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.126:41615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260807 18:35:56.314134 27734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:35:56.313942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:41615 (local address 127.11.14.66:55735)
0807 18:35:56.314045 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0807 18:35:56.314087 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.126:41615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260807 18:35:56.314359 27732 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260807 18:35:56.314373 11321 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkz56XN/build/debug/bin/kudu as pid 27587
I20260807 18:35:56.314781 11321 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkz56XN/build/debug/bin/kudu
/tmp/dist-test-taskkz56XN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.14.67:35171
--local_ip_for_outbound_sockets=127.11.14.67
--tserver_master_addrs=127.11.14.126:41615
--webserver_port=33485
--webserver_interface=127.11.14.67
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.67.keytab
--principal=kudu/127.11.14.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.14.84:39981
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:35:56.434798 27735 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:35:56.435094 27735 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:35:56.435144 27735 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:35:56.439412 27735 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:35:56.439489 27735 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:35:56.439568 27735 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.14.67
I20260807 18:35:56.444890 27735 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.14.84:39981
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.67.keytab
--principal=kudu/127.11.14.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.14.67:35171
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.14.67
--webserver_port=33485
--tserver_master_addrs=127.11.14.126:41615
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27735
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.14.67
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:35:56.446090 27735 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:35:56.447208 27735 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:35:56.457719 27745 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:35:56.457721 27744 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:35:56.457836 27749 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:35:56.457844 27735 server_base.cc:1034] running on GCE node
I20260807 18:35:56.458500 27735 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:35:56.459211 27735 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:35:56.460386 27735 hybrid_clock.cc:648] HybridClock initialized: now 1786127756460350 us; error 47 us; skew 500 ppm
Aug 07 18:35:56 dist-test-slave-6k22 krb5kdc[26133](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127756, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:35:56.463320 27735 init.cc:377] Logged in from keytab as kudu/127.11.14.67@KRBTEST.COM (short username kudu)
I20260807 18:35:56.464375 27735 webserver.cc:460] Webserver started at http://127.11.14.67:33485/ using document root <none> and password file <none>
I20260807 18:35:56.464996 27735 fs_manager.cc:359] Metadata directory not provided
I20260807 18:35:56.465062 27735 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:35:56.469301 27735 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.002s
I20260807 18:35:56.471807 27756 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:35:56.473217 27735 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 18:35:56.473376 27735 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/minicluster-data/ts-2/data,/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/minicluster-data/ts-2/wal
uuid: "3981583d3760440d863c0aa8b1bfb361"
format_stamp: "Formatted at 2026-08-07 18:35:54 on dist-test-slave-6k22"
server_key: "14005ea34632b5c69f2de8014c948c38"
server_key_iv: "9eec48576c193ae9e9b9ec8ec50e0d22"
server_key_version: "encryptionkey@0"
I20260807 18:35:56.473987 27735 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:35:56.487409 27735 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:35:56.491122 27735 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:35:56.491386 27735 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:35:56.492159 27735 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:35:56.493150 27735 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:35:56.493240 27735 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:35:56.493315 27735 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:35:56.493386 27735 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:35:56.503626 27735 rpc_server.cc:307] RPC server started. Bound to: 127.11.14.67:35171
I20260807 18:35:56.503651 27885 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.14.67:35171 every 8 connection(s)
I20260807 18:35:56.504797 27735 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786127579900523-11321-0/minicluster-data/ts-2/data/info.pb
I20260807 18:35:56.509076 27889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:35:56.507178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:41615 (local address 127.11.14.67:51461)
0807 18:35:56.507738 (+ 560us) negotiation.cc:107] Waiting for socket to connect
0807 18:35:56.507867 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.126:41615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":279,"thread_start_us":130,"threads_started":1}
W20260807 18:35:56.509613 27886 heartbeater.cc:644] Failed to heartbeat to 127.11.14.126:41615 (0 consecutive failures): Network error: Failed to ping master at 127.11.14.126:41615: Client connection negotiation failed: client connection to 127.11.14.126:41615: connect: Connection refused (error 111)
I20260807 18:35:56.510305 27889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:35:56.510050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:41615 (local address 127.11.14.67:43697)
0807 18:35:56.510194 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0807 18:35:56.510253 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.126:41615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260807 18:35:56.510982 27889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:35:56.510769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:41615 (local address 127.11.14.67:49177)
0807 18:35:56.510876 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0807 18:35:56.510936 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.126:41615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260807 18:35:56.511246 27886 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260807 18:35:56.512971 11321 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkz56XN/build/debug/bin/kudu as pid 27735
I20260807 18:35:57.185532 27891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:35:57.185011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:41615 (local address 127.11.14.65:60471)
0807 18:35:57.185310 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0807 18:35:57.185375 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.126:41615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":76,"threads_started":1}
I20260807 18:35:57.315331 27892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:35:57.314908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:41615 (local address 127.11.14.66:57821)
0807 18:35:57.315170 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0807 18:35:57.315225 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.126:41615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":86,"threads_started":1}
I20260807 18:35:57.512257 27893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:35:57.511783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:41615 (local address 127.11.14.67:35273)
0807 18:35:57.512080 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0807 18:35:57.512142 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.126:41615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":73,"threads_started":1}
I20260807 18:35:57.815711 27894 raft_consensus.cc:482] T 94ddb3a94032496fbbf9e18b2bcf0ddf P e015739a99844c14829e0b989348e2f1 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:35:57.815907 27894 raft_consensus.cc:504] T 94ddb3a94032496fbbf9e18b2bcf0ddf P e015739a99844c14829e0b989348e2f1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3981583d3760440d863c0aa8b1bfb361" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 35171 } } peers { permanent_uuid: "5e4cd084ed8d4729827ac3a8507c961d" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 39165 } } peers { permanent_uuid: "e015739a99844c14829e0b989348e2f1" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 38891 } }
I20260807 18:35:57.817044 27894 leader_election.cc:288] T 94ddb3a94032496fbbf9e18b2bcf0ddf P e015739a99844c14829e0b989348e2f1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3981583d3760440d863c0aa8b1bfb361 (127.11.14.67:35171), 5e4cd084ed8d4729827ac3a8507c961d (127.11.14.66:39165)
Aug 07 18:35:57 dist-test-slave-6k22 krb5kdc[26133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127756, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.65@KRBTEST.COM for kudu/127.11.14.66@KRBTEST.COM
Aug 07 18:35:57 dist-test-slave-6k22 krb5kdc[26133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127756, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.65@KRBTEST.COM for kudu/127.11.14.67@KRBTEST.COM
I20260807 18:35:57.832504 27899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:35:57.817602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.65:45051 (local address 127.11.14.66:39165)
0807 18:35:57.817985 (+ 383us) server_negotiation.cc:206] Beginning negotiation
0807 18:35:57.817996 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 18:35:57.818028 (+ 32us) server_negotiation.cc:407] Connection header received
0807 18:35:57.818974 (+ 946us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:35:57.818983 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:35:57.819460 (+ 477us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:35:57.819845 (+ 385us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:35:57.821105 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:35:57.822288 (+ 1183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:35:57.823508 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:35:57.824079 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:35:57.824297 (+ 218us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:35:57.827031 (+ 2734us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:35:57.827036 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:35:57.827049 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:35:57.827078 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:35:57.829121 (+ 2043us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:35:57.829753 (+ 632us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:35:57.829762 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:35:57.829772 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:35:57.829952 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:35:57.830462 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:35:57.830465 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:35:57.830467 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:35:57.831090 (+ 623us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:35:57.831238 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0807 18:35:57.832058 (+ 820us) server_negotiation.cc:299] Negotiation successful
0807 18:35:57.832346 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":278,"thread_start_us":66,"threads_started":1}
I20260807 18:35:57.832485 27896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:35:57.817301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.66:39165 (local address 127.11.14.65:45051)
0807 18:35:57.817610 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0807 18:35:57.817626 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 18:35:57.817747 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:35:57.820372 (+ 2625us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:35:57.820375 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:35:57.820546 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:35:57.820896 (+ 350us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:35:57.820912 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:35:57.822431 (+ 1519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:35:57.822434 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:35:57.823344 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:35:57.823354 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:35:57.824273 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:35:57.824278 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:35:57.824437 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:35:57.825044 (+ 607us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:35:57.825063 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:35:57.826747 (+ 1684us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:35:57.829264 (+ 2517us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:35:57.829271 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:35:57.829285 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:35:57.829606 (+ 321us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:35:57.830166 (+ 560us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:35:57.830169 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:35:57.830170 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:35:57.830308 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:35:57.831268 (+ 960us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:35:57.831278 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:35:57.831666 (+ 388us) client_negotiation.cc:769] Sending connection context
0807 18:35:57.831915 (+ 249us) client_negotiation.cc:240] Negotiation successful
0807 18:35:57.832295 (+ 380us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":245,"mutex_wait_us":114,"thread_start_us":71,"threads_started":1}
I20260807 18:35:57.832639 27895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:35:57.817275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.67:35171 (local address 127.11.14.65:57089)
0807 18:35:57.817610 (+ 335us) negotiation.cc:107] Waiting for socket to connect
0807 18:35:57.817626 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 18:35:57.817735 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:35:57.819939 (+ 2204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:35:57.819944 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:35:57.820352 (+ 408us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:35:57.820898 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:35:57.820912 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:35:57.822288 (+ 1376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:35:57.822293 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:35:57.823331 (+ 1038us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:35:57.823342 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:35:57.824208 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:35:57.824212 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:35:57.824438 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:35:57.825229 (+ 791us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:35:57.825243 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:35:57.827167 (+ 1924us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:35:57.829979 (+ 2812us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:35:57.829986 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:35:57.829991 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:35:57.830269 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:35:57.830685 (+ 416us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:35:57.830688 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:35:57.830691 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:35:57.830787 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:35:57.831676 (+ 889us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:35:57.831682 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:35:57.831807 (+ 125us) client_negotiation.cc:769] Sending connection context
0807 18:35:57.831967 (+ 160us) client_negotiation.cc:240] Negotiation successful
0807 18:35:57.832401 (+ 434us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"mutex_wait_us":259,"spinlock_wait_cycles":11136,"thread_start_us":88,"threads_started":1}
I20260807 18:35:57.833251 27898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:35:57.817470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.65:57089 (local address 127.11.14.67:35171)
0807 18:35:57.817940 (+ 470us) server_negotiation.cc:206] Beginning negotiation
0807 18:35:57.817969 (+ 29us) server_negotiation.cc:399] Waiting for connection header
0807 18:35:57.818018 (+ 49us) server_negotiation.cc:407] Connection header received
0807 18:35:57.818974 (+ 956us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:35:57.818983 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:35:57.819460 (+ 477us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:35:57.819845 (+ 385us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:35:57.821132 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:35:57.822125 (+ 993us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:35:57.823476 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:35:57.824046 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:35:57.824207 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:35:57.827392 (+ 3185us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:35:57.827402 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:35:57.827419 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:35:57.827459 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:35:57.829832 (+ 2373us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:35:57.830404 (+ 572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:35:57.830413 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:35:57.830420 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:35:57.830549 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:35:57.830913 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:35:57.830917 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:35:57.830919 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:35:57.831466 (+ 547us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:35:57.831661 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0807 18:35:57.832124 (+ 463us) server_negotiation.cc:299] Negotiation successful
0807 18:35:57.832446 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":327,"thread_start_us":85,"threads_started":1}
W20260807 18:35:57.835664 27460 leader_election.cc:341] T 94ddb3a94032496fbbf9e18b2bcf0ddf P e015739a99844c14829e0b989348e2f1 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5e4cd084ed8d4729827ac3a8507c961d (127.11.14.66:39165): Not found: Tablet not found: 94ddb3a94032496fbbf9e18b2bcf0ddf
W20260807 18:35:57.835897 27461 leader_election.cc:341] T 94ddb3a94032496fbbf9e18b2bcf0ddf P e015739a99844c14829e0b989348e2f1 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3981583d3760440d863c0aa8b1bfb361 (127.11.14.67:35171): Not found: Tablet not found: 94ddb3a94032496fbbf9e18b2bcf0ddf
I20260807 18:35:57.836000 27461 leader_election.cc:302] T 94ddb3a94032496fbbf9e18b2bcf0ddf P e015739a99844c14829e0b989348e2f1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e015739a99844c14829e0b989348e2f1; no voters: 3981583d3760440d863c0aa8b1bfb361, 5e4cd084ed8d4729827ac3a8507c961d
I20260807 18:35:57.836285 27894 raft_consensus.cc:2726] T 94ddb3a94032496fbbf9e18b2bcf0ddf P e015739a99844c14829e0b989348e2f1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 18:35:58.186726 27895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:35:58.186338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:41615 (local address 127.11.14.65:56847)
0807 18:35:58.186527 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0807 18:35:58.186590 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.126:41615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
I20260807 18:35:58.316533 27900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:35:58.316079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:41615 (local address 127.11.14.66:35269)
0807 18:35:58.316361 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0807 18:35:58.316418 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.126:41615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":94,"threads_started":1}
I20260807 18:35:58.513720 27901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:35:58.513166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:41615 (local address 127.11.14.67:43141)
0807 18:35:58.513513 (+ 347us) negotiation.cc:107] Waiting for socket to connect
0807 18:35:58.513568 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.126:41615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":85,"threads_started":1}
I20260807 18:35:59.188089 27902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:35:59.187604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:41615 (local address 127.11.14.65:59523)
0807 18:35:59.187905 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0807 18:35:59.187982 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.126:41615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":80,"threads_started":1}
I20260807 18:35:59.317915 27903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:35:59.317367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:41615 (local address 127.11.14.66:40613)
0807 18:35:59.317729 (+ 362us) negotiation.cc:107] Waiting for socket to connect
0807 18:35:59.317806 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.126:41615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":78,"threads_started":1}
I20260807 18:35:59.515108 27904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:35:59.514593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:41615 (local address 127.11.14.67:42341)
0807 18:35:59.514917 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0807 18:35:59.514978 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.126:41615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":101,"threads_started":1}
I20260807 18:35:59.996101 27906 raft_consensus.cc:482] T 94ddb3a94032496fbbf9e18b2bcf0ddf P e015739a99844c14829e0b989348e2f1 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:35:59.996312 27906 raft_consensus.cc:504] T 94ddb3a94032496fbbf9e18b2bcf0ddf P e015739a99844c14829e0b989348e2f1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3981583d3760440d863c0aa8b1bfb361" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 35171 } } peers { permanent_uuid: "5e4cd084ed8d4729827ac3a8507c961d" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 39165 } } peers { permanent_uuid: "e015739a99844c14829e0b989348e2f1" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 38891 } }
I20260807 18:35:59.996867 27906 leader_election.cc:288] T 94ddb3a94032496fbbf9e18b2bcf0ddf P e015739a99844c14829e0b989348e2f1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3981583d3760440d863c0aa8b1bfb361 (127.11.14.67:35171), 5e4cd084ed8d4729827ac3a8507c961d (127.11.14.66:39165)
W20260807 18:35:59.997772 27461 leader_election.cc:341] T 94ddb3a94032496fbbf9e18b2bcf0ddf P e015739a99844c14829e0b989348e2f1 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3981583d3760440d863c0aa8b1bfb361 (127.11.14.67:35171): Not found: Tablet not found: 94ddb3a94032496fbbf9e18b2bcf0ddf
W20260807 18:35:59.997869 27460 leader_election.cc:341] T 94ddb3a94032496fbbf9e18b2bcf0ddf P e015739a99844c14829e0b989348e2f1 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5e4cd084ed8d4729827ac3a8507c961d (127.11.14.66:39165): Not found: Tablet not found: 94ddb3a94032496fbbf9e18b2bcf0ddf
I20260807 18:35:59.997941 27460 leader_election.cc:302] T 94ddb3a94032496fbbf9e18b2bcf0ddf P e015739a99844c14829e0b989348e2f1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e015739a99844c14829e0b989348e2f1; no voters: 3981583d3760440d863c0aa8b1bfb361, 5e4cd084ed8d4729827ac3a8507c961d
I20260807 18:35:59.998090 27906 raft_consensus.cc:2726] T 94ddb3a94032496fbbf9e18b2bcf0ddf P e015739a99844c14829e0b989348e2f1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 18:36:00.189379 27907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:36:00.188910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:41615 (local address 127.11.14.65:42611)
0807 18:36:00.189196 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0807 18:36:00.189262 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.126:41615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":100,"threads_started":1}
I20260807 18:36:00.319365 27908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:36:00.318808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:41615 (local address 127.11.14.66:45815)
0807 18:36:00.319120 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0807 18:36:00.319209 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.126:41615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":149,"threads_started":1}
I20260807 18:36:00.516752 27909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:36:00.516173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:41615 (local address 127.11.14.67:36823)
0807 18:36:00.516538 (+ 365us) negotiation.cc:107] Waiting for socket to connect
0807 18:36:00.516619 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.126:41615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":81,"threads_started":1}
I20260807 18:36:01.191087 27910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:36:01.190536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:41615 (local address 127.11.14.65:43047)
0807 18:36:01.190912 (+ 376us) negotiation.cc:107] Waiting for socket to connect
0807 18:36:01.190982 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.126:41615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":291,"thread_start_us":86,"threads_started":1}
I20260807 18:36:01.320715 27913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:36:01.320276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:41615 (local address 127.11.14.66:57639)
0807 18:36:01.320554 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0807 18:36:01.320616 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.126:41615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":78,"threads_started":1}
I20260807 18:36:01.518105 27914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:36:01.517670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:41615 (local address 127.11.14.67:49101)
0807 18:36:01.517942 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0807 18:36:01.518003 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.126:41615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":91,"threads_started":1}
I20260807 18:36:02.192555 27915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:36:02.192091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:41615 (local address 127.11.14.65:41277)
0807 18:36:02.192385 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0807 18:36:02.192449 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.126:41615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":90,"threads_started":1}
I20260807 18:36:02.321936 27916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:36:02.321545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:41615 (local address 127.11.14.66:54809)
0807 18:36:02.321786 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0807 18:36:02.321840 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.126:41615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":78,"threads_started":1}
I20260807 18:36:02.519284 27917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:36:02.518889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:41615 (local address 127.11.14.67:42527)
0807 18:36:02.519130 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0807 18:36:02.519187 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.126:41615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":74,"threads_started":1}
I20260807 18:36:03.011477 27918 raft_consensus.cc:482] T 94ddb3a94032496fbbf9e18b2bcf0ddf P e015739a99844c14829e0b989348e2f1 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:36:03.011888 27918 raft_consensus.cc:504] T 94ddb3a94032496fbbf9e18b2bcf0ddf P e015739a99844c14829e0b989348e2f1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3981583d3760440d863c0aa8b1bfb361" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 35171 } } peers { permanent_uuid: "5e4cd084ed8d4729827ac3a8507c961d" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 39165 } } peers { permanent_uuid: "e015739a99844c14829e0b989348e2f1" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 38891 } }
I20260807 18:36:03.012372 27918 leader_election.cc:288] T 94ddb3a94032496fbbf9e18b2bcf0ddf P e015739a99844c14829e0b989348e2f1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3981583d3760440d863c0aa8b1bfb361 (127.11.14.67:35171), 5e4cd084ed8d4729827ac3a8507c961d (127.11.14.66:39165)
W20260807 18:36:03.013347 27461 leader_election.cc:341] T 94ddb3a94032496fbbf9e18b2bcf0ddf P e015739a99844c14829e0b989348e2f1 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3981583d3760440d863c0aa8b1bfb361 (127.11.14.67:35171): Not found: Tablet not found: 94ddb3a94032496fbbf9e18b2bcf0ddf
W20260807 18:36:03.013460 27460 leader_election.cc:341] T 94ddb3a94032496fbbf9e18b2bcf0ddf P e015739a99844c14829e0b989348e2f1 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5e4cd084ed8d4729827ac3a8507c961d (127.11.14.66:39165): Not found: Tablet not found: 94ddb3a94032496fbbf9e18b2bcf0ddf
I20260807 18:36:03.013573 27460 leader_election.cc:302] T 94ddb3a94032496fbbf9e18b2bcf0ddf P e015739a99844c14829e0b989348e2f1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e015739a99844c14829e0b989348e2f1; no voters: 3981583d3760440d863c0aa8b1bfb361, 5e4cd084ed8d4729827ac3a8507c961d
I20260807 18:36:03.013753 27918 raft_consensus.cc:2726] T 94ddb3a94032496fbbf9e18b2bcf0ddf P e015739a99844c14829e0b989348e2f1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 18:36:03.193856 27920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:36:03.193406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:41615 (local address 127.11.14.65:35469)
0807 18:36:03.193679 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0807 18:36:03.193752 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.126:41615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":79,"threads_started":1}
I20260807 18:36:03.323622 27921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:36:03.322982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:41615 (local address 127.11.14.66:41139)
0807 18:36:03.323383 (+ 401us) negotiation.cc:107] Waiting for socket to connect
0807 18:36:03.323446 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.126:41615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":307,"thread_start_us":89,"threads_started":1}
I20260807 18:36:03.520546 27922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:36:03.520081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:41615 (local address 127.11.14.67:44865)
0807 18:36:03.520384 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0807 18:36:03.520448 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.126:41615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":72,"threads_started":1}
I20260807 18:36:04.195016 27923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:36:04.194618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:41615 (local address 127.11.14.65:58197)
0807 18:36:04.194865 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0807 18:36:04.194918 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.126:41615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":79,"threads_started":1}
I20260807 18:36:04.324939 27924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:36:04.324507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:41615 (local address 127.11.14.66:59923)
0807 18:36:04.324763 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0807 18:36:04.324835 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.126:41615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":79,"threads_started":1}
I20260807 18:36:04.522110 27925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:36:04.521434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:41615 (local address 127.11.14.67:41103)
0807 18:36:04.521895 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0807 18:36:04.521972 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.126:41615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":349,"thread_start_us":124,"threads_started":1}
I20260807 18:36:05.196293 27927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:36:05.195864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:41615 (local address 127.11.14.65:47093)
0807 18:36:05.196126 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0807 18:36:05.196186 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.126:41615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":75,"threads_started":1}
I20260807 18:36:05.326146 27928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:36:05.325730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:41615 (local address 127.11.14.66:55747)
0807 18:36:05.326000 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0807 18:36:05.326051 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.126:41615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":87,"threads_started":1}
I20260807 18:36:05.523496 27929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:36:05.522938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:41615 (local address 127.11.14.67:54795)
0807 18:36:05.523272 (+ 334us) negotiation.cc:107] Waiting for socket to connect
0807 18:36:05.523360 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.126:41615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":107,"threads_started":1}
I20260807 18:36:06.197810 27930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:36:06.197364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:41615 (local address 127.11.14.65:53735)
0807 18:36:06.197654 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0807 18:36:06.197712 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.126:41615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":96,"threads_started":1}
I20260807 18:36:06.327521 27931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:36:06.327059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:41615 (local address 127.11.14.66:55747)
0807 18:36:06.327357 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0807 18:36:06.327421 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.126:41615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":86,"threads_started":1}
I20260807 18:36:06.524893 27932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:36:06.524413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:41615 (local address 127.11.14.67:34457)
0807 18:36:06.524711 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0807 18:36:06.524789 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.126:41615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":81,"threads_started":1}
I20260807 18:36:07.199106 27933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:36:07.198664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:41615 (local address 127.11.14.65:34363)
0807 18:36:07.198938 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0807 18:36:07.198996 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.126:41615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":81,"threads_started":1}
I20260807 18:36:07.328830 27934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:36:07.328418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:41615 (local address 127.11.14.66:48945)
0807 18:36:07.328667 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0807 18:36:07.328727 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.126:41615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":77,"threads_started":1}
I20260807 18:36:07.526362 27936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:36:07.525857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:41615 (local address 127.11.14.67:51653)
0807 18:36:07.526164 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0807 18:36:07.526247 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.126:41615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":109,"threads_started":1}
I20260807 18:36:08.053190 27937 raft_consensus.cc:482] T 94ddb3a94032496fbbf9e18b2bcf0ddf P e015739a99844c14829e0b989348e2f1 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:36:08.053368 27937 raft_consensus.cc:504] T 94ddb3a94032496fbbf9e18b2bcf0ddf P e015739a99844c14829e0b989348e2f1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3981583d3760440d863c0aa8b1bfb361" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 35171 } } peers { permanent_uuid: "5e4cd084ed8d4729827ac3a8507c961d" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 39165 } } peers { permanent_uuid: "e015739a99844c14829e0b989348e2f1" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 38891 } }
I20260807 18:36:08.053892 27937 leader_election.cc:288] T 94ddb3a94032496fbbf9e18b2bcf0ddf P e015739a99844c14829e0b989348e2f1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3981583d3760440d863c0aa8b1bfb361 (127.11.14.67:35171), 5e4cd084ed8d4729827ac3a8507c961d (127.11.14.66:39165)
W20260807 18:36:08.054720 27460 leader_election.cc:341] T 94ddb3a94032496fbbf9e18b2bcf0ddf P e015739a99844c14829e0b989348e2f1 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5e4cd084ed8d4729827ac3a8507c961d (127.11.14.66:39165): Not found: Tablet not found: 94ddb3a94032496fbbf9e18b2bcf0ddf
W20260807 18:36:08.054826 27461 leader_election.cc:341] T 94ddb3a94032496fbbf9e18b2bcf0ddf P e015739a99844c14829e0b989348e2f1 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3981583d3760440d863c0aa8b1bfb361 (127.11.14.67:35171): Not found: Tablet not found: 94ddb3a94032496fbbf9e18b2bcf0ddf
I20260807 18:36:08.054947 27461 leader_election.cc:302] T 94ddb3a94032496fbbf9e18b2bcf0ddf P e015739a99844c14829e0b989348e2f1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e015739a99844c14829e0b989348e2f1; no voters: 3981583d3760440d863c0aa8b1bfb361, 5e4cd084ed8d4729827ac3a8507c961d
I20260807 18:36:08.055105 27937 raft_consensus.cc:2726] T 94ddb3a94032496fbbf9e18b2bcf0ddf P e015739a99844c14829e0b989348e2f1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 18:36:08.200390 27938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:36:08.199950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:41615 (local address 127.11.14.65:55719)
0807 18:36:08.200226 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0807 18:36:08.200287 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.126:41615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":82,"threads_started":1}
I20260807 18:36:08.330219 27939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:36:08.329767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:41615 (local address 127.11.14.66:46053)
0807 18:36:08.330043 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0807 18:36:08.330116 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.126:41615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":91,"threads_started":1}
I20260807 18:36:08.527726 27940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:36:08.527139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:41615 (local address 127.11.14.67:53949)
0807 18:36:08.527425 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0807 18:36:08.527489 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.126:41615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":110,"threads_started":1}
I20260807 18:36:09.201779 27941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:36:09.201255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:41615 (local address 127.11.14.65:59463)
0807 18:36:09.201554 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0807 18:36:09.201618 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.126:41615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":83,"threads_started":1}
I20260807 18:36:09.331612 27942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:36:09.331051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:41615 (local address 127.11.14.66:37077)
0807 18:36:09.331379 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0807 18:36:09.331446 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.126:41615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":227,"thread_start_us":88,"threads_started":1}
I20260807 18:36:09.528950 27944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:36:09.528463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:41615 (local address 127.11.14.67:42747)
0807 18:36:09.528791 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0807 18:36:09.528850 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.126:41615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":88,"threads_started":1}
I20260807 18:36:10.202907 27945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:36:10.202518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:41615 (local address 127.11.14.65:52121)
0807 18:36:10.202756 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0807 18:36:10.202809 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.126:41615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":73,"threads_started":1}
I20260807 18:36:10.332940 27946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:36:10.332466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:41615 (local address 127.11.14.66:48399)
0807 18:36:10.332746 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0807 18:36:10.332835 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.126:41615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":103,"threads_started":1}
I20260807 18:36:10.530413 27947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:36:10.529899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:41615 (local address 127.11.14.67:58139)
0807 18:36:10.530245 (+ 346us) negotiation.cc:107] Waiting for socket to connect
0807 18:36:10.530311 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.126:41615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":106,"threads_started":1}
I20260807 18:36:11.204540 27948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:36:11.204103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:41615 (local address 127.11.14.65:54341)
0807 18:36:11.204383 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0807 18:36:11.204441 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.126:41615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":83,"threads_started":1}
I20260807 18:36:11.334288 27951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:36:11.333827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:41615 (local address 127.11.14.66:49087)
0807 18:36:11.334109 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0807 18:36:11.334184 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.126:41615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":80,"threads_started":1}
I20260807 18:36:11.521042 11321 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkz56XN/build/debug/bin/kudu with pid 27403
I20260807 18:36:11.532115 27952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:36:11.531646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:41615 (local address 127.11.14.67:34957)
0807 18:36:11.531945 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0807 18:36:11.532008 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.126:41615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":96,"threads_started":1}
I20260807 18:36:11.537194 11321 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkz56XN/build/debug/bin/kudu with pid 27587
I20260807 18:36:11.545637 11321 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkz56XN/build/debug/bin/kudu with pid 27735
2026-08-07T18:36:11Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (19524 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:36:11 dist-test-slave-6k22 krb5kdc[27955](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:36:11 dist-test-slave-6k22 krb5kdc[27955](info): set up 2 sockets
Aug 07 18:36:11 dist-test-slave-6k22 krb5kdc[27955](info): commencing operation
krb5kdc: starting...
W20260807 18:36:12.724133 11321 mini_kdc.cc:121] Time spent starting KDC: real 1.154s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/krb5kdc/test-user.keytab.
Aug 07 18:36:12 dist-test-slave-6k22 krb5kdc[27955](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127772, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:36:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:36:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.14.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.14.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.14.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.126.keytab.
Entry for principal kudu/127.11.14.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.126.keytab.
I20260807 18:36:12.859695 11321 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkz56XN/build/debug/bin/kudu
/tmp/dist-test-taskkz56XN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.14.126:40109
--webserver_interface=127.11.14.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.126.keytab
--principal=kudu/127.11.14.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.14.84:35419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.14.126:40109
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:36:12.976857 27968 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:36:12.977187 27968 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:36:12.977263 27968 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:36:12.981307 27968 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260807 18:36:12.981400 27968 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260807 18:36:12.981426 27968 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260807 18:36:12.981448 27968 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:36:12.981492 27968 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:36:12.981534 27968 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:36:12.981595 27968 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:36:12.981633 27968 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:36:12.987268 27968 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.14.84:35419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.11.14.126:40109
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.126.keytab
--principal=kudu/127.11.14.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.14.126:40109
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.14.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27968
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:36:12.988515 27968 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:36:12.989624 27968 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 18:36:12.998842 27968 server_base.cc:1034] running on GCE node
W20260807 18:36:12.998912 27979 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:36:12.998925 27976 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:36:12.999009 27975 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:36:12.999770 27968 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:36:13.000962 27968 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:36:13.002161 27968 hybrid_clock.cc:648] HybridClock initialized: now 1786127773002141 us; error 32 us; skew 500 ppm
Aug 07 18:36:13 dist-test-slave-6k22 krb5kdc[27955](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127773, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:36:13.005091 27968 init.cc:377] Logged in from keytab as kudu/127.11.14.126@KRBTEST.COM (short username kudu)
I20260807 18:36:13.006033 27968 webserver.cc:460] Webserver started at http://127.11.14.126:41133/ using document root <none> and password file <none>
I20260807 18:36:13.006625 27968 fs_manager.cc:359] Metadata directory not provided
I20260807 18:36:13.006721 27968 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:36:13.006968 27968 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:36:13.009128 27968 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/minicluster-data/master-0/data/instance:
uuid: "9e30e67c1f6d4fb68e8f67b10bc50f96"
format_stamp: "Formatted at 2026-08-07 18:36:13 on dist-test-slave-6k22"
server_key: "a80393781a76b348478afae6879517bd"
server_key_iv: "de5d6f34fc2b7da1a136cd32c644d014"
server_key_version: "encryptionkey@0"
I20260807 18:36:13.009725 27968 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/minicluster-data/master-0/wal/instance:
uuid: "9e30e67c1f6d4fb68e8f67b10bc50f96"
format_stamp: "Formatted at 2026-08-07 18:36:13 on dist-test-slave-6k22"
server_key: "a80393781a76b348478afae6879517bd"
server_key_iv: "de5d6f34fc2b7da1a136cd32c644d014"
server_key_version: "encryptionkey@0"
I20260807 18:36:13.013671 27968 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 18:36:13.016307 27986 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:36:13.017871 27968 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:36:13.018049 27968 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/minicluster-data/master-0/data,/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/minicluster-data/master-0/wal
uuid: "9e30e67c1f6d4fb68e8f67b10bc50f96"
format_stamp: "Formatted at 2026-08-07 18:36:13 on dist-test-slave-6k22"
server_key: "a80393781a76b348478afae6879517bd"
server_key_iv: "de5d6f34fc2b7da1a136cd32c644d014"
server_key_version: "encryptionkey@0"
I20260807 18:36:13.018190 27968 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:36:13.039922 27968 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:36:13.043131 27968 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:36:13.043367 27968 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:36:13.050797 27968 rpc_server.cc:307] RPC server started. Bound to: 127.11.14.126:40109
I20260807 18:36:13.051110 28046 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.14.126:40109 every 8 connection(s)
I20260807 18:36:13.051916 27968 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/minicluster-data/master-0/data/info.pb
I20260807 18:36:13.054840 28047 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:36:13.057242 11321 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkz56XN/build/debug/bin/kudu as pid 27968
I20260807 18:36:13.057403 11321 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/minicluster-data/master-0/wal/instance
I20260807 18:36:13.057695 11321 external_mini_cluster.cc:1428] Setting key 8229b952305c99626da0d0ccadbf3d97
I20260807 18:36:13.061565 28047 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9e30e67c1f6d4fb68e8f67b10bc50f96: Bootstrap starting.
I20260807 18:36:13.064128 28047 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9e30e67c1f6d4fb68e8f67b10bc50f96: Neither blocks nor log segments found. Creating new log.
I20260807 18:36:13.064958 28047 log.cc:824] T 00000000000000000000000000000000 P 9e30e67c1f6d4fb68e8f67b10bc50f96: Log is configured to *not* fsync() on all Append() calls
I20260807 18:36:13.067495 28047 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9e30e67c1f6d4fb68e8f67b10bc50f96: No bootstrap required, opened a new log
Aug 07 18:36:13 dist-test-slave-6k22 krb5kdc[27955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127772, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.14.126@KRBTEST.COM
I20260807 18:36:13.072541 28047 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9e30e67c1f6d4fb68e8f67b10bc50f96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e30e67c1f6d4fb68e8f67b10bc50f96" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 40109 } }
I20260807 18:36:13.072839 28047 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9e30e67c1f6d4fb68e8f67b10bc50f96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:36:13.072921 28047 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9e30e67c1f6d4fb68e8f67b10bc50f96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e30e67c1f6d4fb68e8f67b10bc50f96, State: Initialized, Role: FOLLOWER
I20260807 18:36:13.073432 28047 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9e30e67c1f6d4fb68e8f67b10bc50f96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e30e67c1f6d4fb68e8f67b10bc50f96" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 40109 } }
I20260807 18:36:13.073592 28047 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9e30e67c1f6d4fb68e8f67b10bc50f96 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:36:13.073671 28047 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9e30e67c1f6d4fb68e8f67b10bc50f96 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:36:13.073768 28047 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9e30e67c1f6d4fb68e8f67b10bc50f96 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:36:13.073963 28050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:36:13.059228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46252 (local address 127.11.14.126:40109)
0807 18:36:13.059771 (+ 543us) server_negotiation.cc:206] Beginning negotiation
0807 18:36:13.059789 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0807 18:36:13.059826 (+ 37us) server_negotiation.cc:407] Connection header received
0807 18:36:13.060535 (+ 709us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:36:13.060554 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:36:13.060900 (+ 346us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:36:13.061194 (+ 294us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:36:13.062153 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:36:13.063321 (+ 1168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:36:13.064571 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:36:13.065249 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:36:13.065453 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:36:13.068377 (+ 2924us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:36:13.068384 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:36:13.068402 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:36:13.068433 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:36:13.070624 (+ 2191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:36:13.071111 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:36:13.071117 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:36:13.071123 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:36:13.071284 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:36:13.071548 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:36:13.071551 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:36:13.071553 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:36:13.072142 (+ 589us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:36:13.072348 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0807 18:36:13.072605 (+ 257us) server_negotiation.cc:299] Negotiation successful
0807 18:36:13.072942 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":393,"thread_start_us":251,"threads_started":1}
I20260807 18:36:13.074983 28047 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9e30e67c1f6d4fb68e8f67b10bc50f96 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e30e67c1f6d4fb68e8f67b10bc50f96" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 40109 } }
I20260807 18:36:13.075392 28047 leader_election.cc:302] T 00000000000000000000000000000000 P 9e30e67c1f6d4fb68e8f67b10bc50f96 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9e30e67c1f6d4fb68e8f67b10bc50f96; no voters:
I20260807 18:36:13.075762 28047 leader_election.cc:288] T 00000000000000000000000000000000 P 9e30e67c1f6d4fb68e8f67b10bc50f96 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:36:13.075850 28052 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9e30e67c1f6d4fb68e8f67b10bc50f96 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:36:13.076081 28052 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9e30e67c1f6d4fb68e8f67b10bc50f96 [term 1 LEADER]: Becoming Leader. State: Replica: 9e30e67c1f6d4fb68e8f67b10bc50f96, State: Running, Role: LEADER
I20260807 18:36:13.076443 28052 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9e30e67c1f6d4fb68e8f67b10bc50f96 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e30e67c1f6d4fb68e8f67b10bc50f96" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 40109 } }
I20260807 18:36:13.077167 28047 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9e30e67c1f6d4fb68e8f67b10bc50f96 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:36:13.077749 28052 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9e30e67c1f6d4fb68e8f67b10bc50f96 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9e30e67c1f6d4fb68e8f67b10bc50f96. Latest consensus state: current_term: 1 leader_uuid: "9e30e67c1f6d4fb68e8f67b10bc50f96" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e30e67c1f6d4fb68e8f67b10bc50f96" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 40109 } } }
I20260807 18:36:13.077965 28052 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9e30e67c1f6d4fb68e8f67b10bc50f96 [sys.catalog]: This master's current role is: LEADER
I20260807 18:36:13.078441 28059 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:36:13.078488 28053 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9e30e67c1f6d4fb68e8f67b10bc50f96 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9e30e67c1f6d4fb68e8f67b10bc50f96" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e30e67c1f6d4fb68e8f67b10bc50f96" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 40109 } } }
I20260807 18:36:13.078703 28053 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9e30e67c1f6d4fb68e8f67b10bc50f96 [sys.catalog]: This master's current role is: LEADER
I20260807 18:36:13.081688 28059 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:36:13.087364 28059 catalog_manager.cc:1347] Generated new cluster ID: 6f64b49830594df991154b1ee8acec8f
I20260807 18:36:13.087486 28059 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:36:13.104171 28059 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:36:13.105188 28059 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:36:13.112443 28059 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9e30e67c1f6d4fb68e8f67b10bc50f96: Generated new TSK 0
I20260807 18:36:13.113189 28059 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.14.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.14.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.14.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.65.keytab.
Entry for principal kudu/127.11.14.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.65.keytab.
I20260807 18:36:13.151124 11321 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkz56XN/build/debug/bin/kudu
/tmp/dist-test-taskkz56XN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.14.65:0
--local_ip_for_outbound_sockets=127.11.14.65
--webserver_interface=127.11.14.65
--webserver_port=0
--tserver_master_addrs=127.11.14.126:40109
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.65.keytab
--principal=kudu/127.11.14.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.14.84:35419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:36:13.269961 28075 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:36:13.270290 28075 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:36:13.270335 28075 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:36:13.274492 28075 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:36:13.274570 28075 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:36:13.274631 28075 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.14.65
I20260807 18:36:13.280551 28075 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.14.84:35419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.65.keytab
--principal=kudu/127.11.14.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.14.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.14.65
--webserver_port=0
--tserver_master_addrs=127.11.14.126:40109
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28075
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.14.65
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:36:13.281747 28075 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:36:13.282838 28075 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:36:13.293023 28082 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:36:13.293076 28081 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:36:13.293062 28084 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:36:13.293370 28075 server_base.cc:1034] running on GCE node
I20260807 18:36:13.293800 28075 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:36:13.294448 28075 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:36:13.295710 28075 hybrid_clock.cc:648] HybridClock initialized: now 1786127773295614 us; error 37 us; skew 500 ppm
Aug 07 18:36:13 dist-test-slave-6k22 krb5kdc[27955](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127773, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:36:13.298940 28075 init.cc:377] Logged in from keytab as kudu/127.11.14.65@KRBTEST.COM (short username kudu)
I20260807 18:36:13.300053 28075 webserver.cc:460] Webserver started at http://127.11.14.65:34441/ using document root <none> and password file <none>
I20260807 18:36:13.300685 28075 fs_manager.cc:359] Metadata directory not provided
I20260807 18:36:13.300784 28075 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:36:13.301043 28075 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:36:13.303277 28075 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/minicluster-data/ts-0/data/instance:
uuid: "8a890d9b2c9d44be9139631c5bdb2967"
format_stamp: "Formatted at 2026-08-07 18:36:13 on dist-test-slave-6k22"
server_key: "1ff1f02b7e55fca2ea09aa675715ea75"
server_key_iv: "c3fe88fa8a4dabab522d6eec85ec86e9"
server_key_version: "encryptionkey@0"
I20260807 18:36:13.303921 28075 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/minicluster-data/ts-0/wal/instance:
uuid: "8a890d9b2c9d44be9139631c5bdb2967"
format_stamp: "Formatted at 2026-08-07 18:36:13 on dist-test-slave-6k22"
server_key: "1ff1f02b7e55fca2ea09aa675715ea75"
server_key_iv: "c3fe88fa8a4dabab522d6eec85ec86e9"
server_key_version: "encryptionkey@0"
I20260807 18:36:13.308106 28075 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260807 18:36:13.310845 28092 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:36:13.312301 28075 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:36:13.312474 28075 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/minicluster-data/ts-0/data,/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/minicluster-data/ts-0/wal
uuid: "8a890d9b2c9d44be9139631c5bdb2967"
format_stamp: "Formatted at 2026-08-07 18:36:13 on dist-test-slave-6k22"
server_key: "1ff1f02b7e55fca2ea09aa675715ea75"
server_key_iv: "c3fe88fa8a4dabab522d6eec85ec86e9"
server_key_version: "encryptionkey@0"
I20260807 18:36:13.312605 28075 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:36:13.333786 28075 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:36:13.337750 28075 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:36:13.337987 28075 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:36:13.338702 28075 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:36:13.339668 28075 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:36:13.339745 28075 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:36:13.339824 28075 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:36:13.339880 28075 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:36:13.349880 28075 rpc_server.cc:307] RPC server started. Bound to: 127.11.14.65:45483
I20260807 18:36:13.349906 28212 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.14.65:45483 every 8 connection(s)
I20260807 18:36:13.350878 28075 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/minicluster-data/ts-0/data/info.pb
I20260807 18:36:13.358929 11321 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkz56XN/build/debug/bin/kudu as pid 28075
I20260807 18:36:13.359084 11321 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/minicluster-data/ts-0/wal/instance
I20260807 18:36:13.359470 11321 external_mini_cluster.cc:1428] Setting key 35dbda01547fd688c023804d7d3fc05f
Aug 07 18:36:13 dist-test-slave-6k22 krb5kdc[27955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127773, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.65@KRBTEST.COM for kudu/127.11.14.126@KRBTEST.COM
I20260807 18:36:13.366478 28050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:36:13.353007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.65:42519 (local address 127.11.14.126:40109)
0807 18:36:13.353154 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0807 18:36:13.353157 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 18:36:13.354285 (+ 1128us) server_negotiation.cc:407] Connection header received
0807 18:36:13.355166 (+ 881us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:36:13.355170 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:36:13.355232 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:36:13.355322 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:36:13.356567 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:36:13.357198 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:36:13.358363 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:36:13.358640 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:36:13.358808 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:36:13.361748 (+ 2940us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:36:13.361756 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:36:13.361762 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:36:13.361798 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:36:13.363790 (+ 1992us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:36:13.364510 (+ 720us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:36:13.364513 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:36:13.364514 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:36:13.364603 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:36:13.365077 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:36:13.365079 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:36:13.365081 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:36:13.365281 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:36:13.365367 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0807 18:36:13.366016 (+ 649us) server_negotiation.cc:299] Negotiation successful
0807 18:36:13.366194 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260807 18:36:13.367094 28215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:36:13.353410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:40109 (local address 127.11.14.65:42519)
0807 18:36:13.354116 (+ 706us) negotiation.cc:107] Waiting for socket to connect
0807 18:36:13.354157 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 18:36:13.354927 (+ 770us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:36:13.355493 (+ 566us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:36:13.355499 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:36:13.355924 (+ 425us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:36:13.356416 (+ 492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:36:13.356430 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:36:13.357319 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:36:13.357322 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:36:13.358210 (+ 888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:36:13.358219 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:36:13.358881 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:36:13.358887 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:36:13.359115 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:36:13.359889 (+ 774us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:36:13.359909 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:36:13.361550 (+ 1641us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:36:13.363934 (+ 2384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:36:13.363943 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:36:13.363966 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:36:13.364372 (+ 406us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:36:13.364765 (+ 393us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:36:13.364770 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:36:13.364775 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:36:13.364975 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:36:13.365395 (+ 420us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:36:13.365404 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:36:13.365714 (+ 310us) client_negotiation.cc:769] Sending connection context
0807 18:36:13.365888 (+ 174us) client_negotiation.cc:240] Negotiation successful
0807 18:36:13.366159 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":512,"thread_start_us":131,"threads_started":1}
I20260807 18:36:13.368554 28213 heartbeater.cc:347] Connected to a master server at 127.11.14.126:40109
I20260807 18:36:13.368829 28213 heartbeater.cc:464] Registering TS with master...
I20260807 18:36:13.369349 28213 heartbeater.cc:511] Master 127.11.14.126:40109 requested a full tablet report, sending...
I20260807 18:36:13.371256 28005 ts_manager.cc:194] Registered new tserver with Master: 8a890d9b2c9d44be9139631c5bdb2967 (127.11.14.65:45483)
WARNING: no policy specified for kudu/127.11.14.66@KRBTEST.COM; defaulting to no policy
I20260807 18:36:13.373087 28005 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.14.65@KRBTEST.COM'} at 127.11.14.65:42519
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.14.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.14.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.66.keytab.
Entry for principal kudu/127.11.14.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.66.keytab.
I20260807 18:36:13.394735 11321 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkz56XN/build/debug/bin/kudu
/tmp/dist-test-taskkz56XN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.14.66:0
--local_ip_for_outbound_sockets=127.11.14.66
--webserver_interface=127.11.14.66
--webserver_port=0
--tserver_master_addrs=127.11.14.126:40109
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.66.keytab
--principal=kudu/127.11.14.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.14.84:35419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:36:13.512858 28218 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:36:13.513132 28218 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:36:13.513177 28218 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:36:13.517207 28218 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:36:13.517266 28218 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:36:13.517324 28218 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.14.66
I20260807 18:36:13.522591 28218 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.14.84:35419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.66.keytab
--principal=kudu/127.11.14.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.14.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.14.66
--webserver_port=0
--tserver_master_addrs=127.11.14.126:40109
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28218
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.14.66
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:36:13.523622 28218 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:36:13.524598 28218 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:36:13.534452 28223 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:36:13.534458 28226 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:36:13.534458 28224 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:36:13.534890 28218 server_base.cc:1034] running on GCE node
I20260807 18:36:13.535382 28218 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:36:13.536116 28218 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:36:13.537290 28218 hybrid_clock.cc:648] HybridClock initialized: now 1786127773537271 us; error 34 us; skew 500 ppm
Aug 07 18:36:13 dist-test-slave-6k22 krb5kdc[27955](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127773, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:36:13.540444 28218 init.cc:377] Logged in from keytab as kudu/127.11.14.66@KRBTEST.COM (short username kudu)
I20260807 18:36:13.541352 28218 webserver.cc:460] Webserver started at http://127.11.14.66:36717/ using document root <none> and password file <none>
I20260807 18:36:13.541950 28218 fs_manager.cc:359] Metadata directory not provided
I20260807 18:36:13.542047 28218 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:36:13.542300 28218 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:36:13.544494 28218 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/minicluster-data/ts-1/data/instance:
uuid: "8dd92e134d474cdfbb667f7435670c8d"
format_stamp: "Formatted at 2026-08-07 18:36:13 on dist-test-slave-6k22"
server_key: "f57f065283ab1a4e5f3b246de1834c43"
server_key_iv: "0653dbc1ced63c01d8f6843d603fe578"
server_key_version: "encryptionkey@0"
I20260807 18:36:13.545076 28218 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/minicluster-data/ts-1/wal/instance:
uuid: "8dd92e134d474cdfbb667f7435670c8d"
format_stamp: "Formatted at 2026-08-07 18:36:13 on dist-test-slave-6k22"
server_key: "f57f065283ab1a4e5f3b246de1834c43"
server_key_iv: "0653dbc1ced63c01d8f6843d603fe578"
server_key_version: "encryptionkey@0"
I20260807 18:36:13.549022 28218 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 18:36:13.551615 28234 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:36:13.552948 28218 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 18:36:13.553092 28218 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/minicluster-data/ts-1/data,/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/minicluster-data/ts-1/wal
uuid: "8dd92e134d474cdfbb667f7435670c8d"
format_stamp: "Formatted at 2026-08-07 18:36:13 on dist-test-slave-6k22"
server_key: "f57f065283ab1a4e5f3b246de1834c43"
server_key_iv: "0653dbc1ced63c01d8f6843d603fe578"
server_key_version: "encryptionkey@0"
I20260807 18:36:13.553220 28218 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:36:13.574803 28218 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:36:13.578795 28218 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:36:13.579020 28218 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:36:13.579784 28218 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:36:13.580690 28218 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:36:13.580765 28218 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:36:13.580838 28218 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:36:13.580892 28218 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:36:13.590184 28218 rpc_server.cc:307] RPC server started. Bound to: 127.11.14.66:39389
I20260807 18:36:13.590211 28358 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.14.66:39389 every 8 connection(s)
I20260807 18:36:13.591127 28218 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/minicluster-data/ts-1/data/info.pb
I20260807 18:36:13.592461 11321 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkz56XN/build/debug/bin/kudu as pid 28218
I20260807 18:36:13.592568 11321 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/minicluster-data/ts-1/wal/instance
I20260807 18:36:13.592870 11321 external_mini_cluster.cc:1428] Setting key df552c78a981306475110e47cba96669
Aug 07 18:36:13 dist-test-slave-6k22 krb5kdc[27955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127773, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.66@KRBTEST.COM for kudu/127.11.14.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.11.14.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.14.67@KRBTEST.COM" created.
I20260807 18:36:13.608836 28050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:36:13.593240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.66:42749 (local address 127.11.14.126:40109)
0807 18:36:13.593392 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0807 18:36:13.593397 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:36:13.594918 (+ 1521us) server_negotiation.cc:407] Connection header received
0807 18:36:13.596512 (+ 1594us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:36:13.596515 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:36:13.596563 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:36:13.596660 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:36:13.598117 (+ 1457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:36:13.599051 (+ 934us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:36:13.600555 (+ 1504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:36:13.600994 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:36:13.601137 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:36:13.604396 (+ 3259us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:36:13.604401 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:36:13.604404 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:36:13.604442 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:36:13.606128 (+ 1686us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:36:13.606775 (+ 647us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:36:13.606781 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:36:13.606783 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:36:13.606884 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:36:13.607315 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:36:13.607318 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:36:13.607320 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:36:13.607552 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:36:13.607840 (+ 288us) server_negotiation.cc:1035] Waiting for connection context
0807 18:36:13.608440 (+ 600us) server_negotiation.cc:299] Negotiation successful
0807 18:36:13.608671 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260807 18:36:13.609753 28361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:36:13.593773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:40109 (local address 127.11.14.66:42749)
0807 18:36:13.594736 (+ 963us) negotiation.cc:107] Waiting for socket to connect
0807 18:36:13.594800 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0807 18:36:13.596253 (+ 1453us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:36:13.596889 (+ 636us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:36:13.596902 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:36:13.597419 (+ 517us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:36:13.597950 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:36:13.597964 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:36:13.599205 (+ 1241us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:36:13.599208 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:36:13.600402 (+ 1194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:36:13.600414 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:36:13.601200 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:36:13.601203 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:36:13.601365 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:36:13.602263 (+ 898us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:36:13.602296 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:36:13.604178 (+ 1882us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:36:13.606249 (+ 2071us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:36:13.606255 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:36:13.606269 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:36:13.606590 (+ 321us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:36:13.607000 (+ 410us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:36:13.607003 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:36:13.607004 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:36:13.607160 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:36:13.607857 (+ 697us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:36:13.607863 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:36:13.608122 (+ 259us) client_negotiation.cc:769] Sending connection context
0807 18:36:13.608336 (+ 214us) client_negotiation.cc:240] Negotiation successful
0807 18:36:13.608763 (+ 427us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":670,"thread_start_us":151,"threads_started":1}
I20260807 18:36:13.611074 28359 heartbeater.cc:347] Connected to a master server at 127.11.14.126:40109
I20260807 18:36:13.611405 28359 heartbeater.cc:464] Registering TS with master...
I20260807 18:36:13.612211 28359 heartbeater.cc:511] Master 127.11.14.126:40109 requested a full tablet report, sending...
I20260807 18:36:13.613484 28005 ts_manager.cc:194] Registered new tserver with Master: 8dd92e134d474cdfbb667f7435670c8d (127.11.14.66:39389)
I20260807 18:36:13.614332 28005 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.14.66@KRBTEST.COM'} at 127.11.14.66:42749
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.14.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.67.keytab.
Entry for principal kudu/127.11.14.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.67.keytab.
I20260807 18:36:13.628067 11321 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkz56XN/build/debug/bin/kudu
/tmp/dist-test-taskkz56XN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.14.67:0
--local_ip_for_outbound_sockets=127.11.14.67
--webserver_interface=127.11.14.67
--webserver_port=0
--tserver_master_addrs=127.11.14.126:40109
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.67.keytab
--principal=kudu/127.11.14.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.14.84:35419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:36:13.741837 28364 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:36:13.742115 28364 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:36:13.742195 28364 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:36:13.746295 28364 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:36:13.746407 28364 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:36:13.746495 28364 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.14.67
I20260807 18:36:13.751760 28364 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.14.84:35419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.67.keytab
--principal=kudu/127.11.14.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.14.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.14.67
--webserver_port=0
--tserver_master_addrs=127.11.14.126:40109
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28364
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.14.67
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:36:13.752869 28364 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:36:13.754009 28364 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:36:13.764117 28369 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:36:13.764161 28372 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:36:13.764143 28370 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:36:13.764338 28364 server_base.cc:1034] running on GCE node
I20260807 18:36:13.764837 28364 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:36:13.765484 28364 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:36:13.766651 28364 hybrid_clock.cc:648] HybridClock initialized: now 1786127773766615 us; error 47 us; skew 500 ppm
Aug 07 18:36:13 dist-test-slave-6k22 krb5kdc[27955](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127773, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:36:13.769486 28364 init.cc:377] Logged in from keytab as kudu/127.11.14.67@KRBTEST.COM (short username kudu)
I20260807 18:36:13.770460 28364 webserver.cc:460] Webserver started at http://127.11.14.67:38235/ using document root <none> and password file <none>
I20260807 18:36:13.771077 28364 fs_manager.cc:359] Metadata directory not provided
I20260807 18:36:13.771173 28364 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:36:13.771420 28364 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:36:13.773562 28364 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/minicluster-data/ts-2/data/instance:
uuid: "f6c9cfee63a24a18b79f2e34c4222e44"
format_stamp: "Formatted at 2026-08-07 18:36:13 on dist-test-slave-6k22"
server_key: "1f168783971fc54e3f339eabb2b63d2b"
server_key_iv: "9d915462a34803bbef932b44281d251e"
server_key_version: "encryptionkey@0"
I20260807 18:36:13.774170 28364 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/minicluster-data/ts-2/wal/instance:
uuid: "f6c9cfee63a24a18b79f2e34c4222e44"
format_stamp: "Formatted at 2026-08-07 18:36:13 on dist-test-slave-6k22"
server_key: "1f168783971fc54e3f339eabb2b63d2b"
server_key_iv: "9d915462a34803bbef932b44281d251e"
server_key_version: "encryptionkey@0"
I20260807 18:36:13.778162 28364 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 18:36:13.780840 28381 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:36:13.782102 28364 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 18:36:13.782239 28364 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/minicluster-data/ts-2/data,/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/minicluster-data/ts-2/wal
uuid: "f6c9cfee63a24a18b79f2e34c4222e44"
format_stamp: "Formatted at 2026-08-07 18:36:13 on dist-test-slave-6k22"
server_key: "1f168783971fc54e3f339eabb2b63d2b"
server_key_iv: "9d915462a34803bbef932b44281d251e"
server_key_version: "encryptionkey@0"
I20260807 18:36:13.782361 28364 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:36:13.797777 28364 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:36:13.800899 28364 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:36:13.801110 28364 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:36:13.801733 28364 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:36:13.802613 28364 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:36:13.802683 28364 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:36:13.802765 28364 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:36:13.802816 28364 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:36:13.812321 28364 rpc_server.cc:307] RPC server started. Bound to: 127.11.14.67:39057
I20260807 18:36:13.812352 28511 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.14.67:39057 every 8 connection(s)
I20260807 18:36:13.813247 28364 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/minicluster-data/ts-2/data/info.pb
I20260807 18:36:13.814841 11321 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkz56XN/build/debug/bin/kudu as pid 28364
I20260807 18:36:13.814996 11321 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/minicluster-data/ts-2/wal/instance
I20260807 18:36:13.815330 11321 external_mini_cluster.cc:1428] Setting key 353cada9bd35ef641519b481989c1701
Aug 07 18:36:13 dist-test-slave-6k22 krb5kdc[27955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127773, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.67@KRBTEST.COM for kudu/127.11.14.126@KRBTEST.COM
I20260807 18:36:13.827190 28050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:36:13.815119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.67:45069 (local address 127.11.14.126:40109)
0807 18:36:13.815300 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0807 18:36:13.815306 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:36:13.816632 (+ 1326us) server_negotiation.cc:407] Connection header received
0807 18:36:13.817818 (+ 1186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:36:13.817821 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:36:13.817879 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:36:13.817954 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:36:13.819158 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:36:13.819768 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:36:13.820934 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:36:13.821154 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:36:13.821268 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:36:13.823815 (+ 2547us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:36:13.823824 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:36:13.823826 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:36:13.823850 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:36:13.825122 (+ 1272us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:36:13.825632 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:36:13.825636 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:36:13.825638 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:36:13.825728 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:36:13.826073 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:36:13.826078 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:36:13.826079 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:36:13.826260 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:36:13.826368 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0807 18:36:13.826890 (+ 522us) server_negotiation.cc:299] Negotiation successful
0807 18:36:13.827014 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260807 18:36:13.828048 28516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:36:13.815654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:40109 (local address 127.11.14.67:45069)
0807 18:36:13.816489 (+ 835us) negotiation.cc:107] Waiting for socket to connect
0807 18:36:13.816536 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0807 18:36:13.817579 (+ 1043us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:36:13.818134 (+ 555us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:36:13.818141 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:36:13.818532 (+ 391us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:36:13.819015 (+ 483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:36:13.819029 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:36:13.819898 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:36:13.819901 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:36:13.820795 (+ 894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:36:13.820806 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:36:13.821342 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:36:13.821348 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:36:13.821525 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:36:13.822137 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:36:13.822155 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:36:13.823654 (+ 1499us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:36:13.825247 (+ 1593us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:36:13.825252 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:36:13.825267 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:36:13.825523 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:36:13.825824 (+ 301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:36:13.825827 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:36:13.825829 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:36:13.825974 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:36:13.826381 (+ 407us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:36:13.826389 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:36:13.826634 (+ 245us) client_negotiation.cc:769] Sending connection context
0807 18:36:13.826833 (+ 199us) client_negotiation.cc:240] Negotiation successful
0807 18:36:13.827163 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":579,"thread_start_us":114,"threads_started":1}
I20260807 18:36:13.829226 28512 heartbeater.cc:347] Connected to a master server at 127.11.14.126:40109
I20260807 18:36:13.829466 28512 heartbeater.cc:464] Registering TS with master...
I20260807 18:36:13.829972 28512 heartbeater.cc:511] Master 127.11.14.126:40109 requested a full tablet report, sending...
I20260807 18:36:13.831014 28005 ts_manager.cc:194] Registered new tserver with Master: f6c9cfee63a24a18b79f2e34c4222e44 (127.11.14.67:39057)
I20260807 18:36:13.831653 28005 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.14.67@KRBTEST.COM'} at 127.11.14.67:45069
I20260807 18:36:13.841348 11321 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 18:36:13.850489 28050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:36:13.843428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46262 (local address 127.11.14.126:40109)
0807 18:36:13.843596 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0807 18:36:13.843607 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 18:36:13.843776 (+ 169us) server_negotiation.cc:407] Connection header received
0807 18:36:13.843885 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:36:13.843888 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:36:13.843930 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:36:13.844002 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:36:13.844736 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:36:13.845355 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:36:13.846200 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:36:13.846415 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:36:13.846498 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:36:13.847817 (+ 1319us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:36:13.847820 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:36:13.847822 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:36:13.847844 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:36:13.849074 (+ 1230us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:36:13.849442 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:36:13.849445 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:36:13.849446 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:36:13.849519 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:36:13.849764 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:36:13.849766 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:36:13.849767 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:36:13.849982 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:36:13.850083 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0807 18:36:13.850280 (+ 197us) server_negotiation.cc:299] Negotiation successful
0807 18:36:13.850371 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260807 18:36:13.853338 28005 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46262:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 18:36:13.856355 28005 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 18:36:13.871132 28529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:36:13.865074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45424 (local address 127.11.14.65:45483)
0807 18:36:13.866278 (+ 1204us) server_negotiation.cc:206] Beginning negotiation
0807 18:36:13.866294 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 18:36:13.866321 (+ 27us) server_negotiation.cc:407] Connection header received
0807 18:36:13.866407 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:36:13.866412 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:36:13.866619 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:36:13.866793 (+ 174us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:36:13.867106 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:36:13.868334 (+ 1228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:36:13.869802 (+ 1468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:36:13.870415 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:36:13.870536 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:36:13.870692 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0807 18:36:13.870880 (+ 188us) server_negotiation.cc:299] Negotiation successful
0807 18:36:13.870974 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1075,"thread_start_us":111,"threads_started":1}
I20260807 18:36:13.871119 28530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:36:13.864943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.65:45483 (local address 127.0.0.1:45424)
0807 18:36:13.865461 (+ 518us) negotiation.cc:107] Waiting for socket to connect
0807 18:36:13.865512 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0807 18:36:13.865651 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:36:13.866802 (+ 1151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:36:13.866810 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:36:13.866832 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:36:13.866991 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:36:13.866998 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:36:13.868700 (+ 1702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:36:13.868704 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:36:13.869677 (+ 973us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:36:13.869689 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:36:13.870549 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:36:13.870553 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:36:13.870665 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:36:13.870705 (+ 40us) client_negotiation.cc:769] Sending connection context
0807 18:36:13.870847 (+ 142us) client_negotiation.cc:240] Negotiation successful
0807 18:36:13.870974 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":346,"thread_start_us":88,"threads_started":1}
I20260807 18:36:13.871712 28533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:36:13.865884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46198 (local address 127.11.14.66:39389)
0807 18:36:13.866323 (+ 439us) server_negotiation.cc:206] Beginning negotiation
0807 18:36:13.866333 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 18:36:13.866354 (+ 21us) server_negotiation.cc:407] Connection header received
0807 18:36:13.866483 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:36:13.866487 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:36:13.866627 (+ 140us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:36:13.866794 (+ 167us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:36:13.867340 (+ 546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:36:13.868453 (+ 1113us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:36:13.869972 (+ 1519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:36:13.870790 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:36:13.870909 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:36:13.871052 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0807 18:36:13.871438 (+ 386us) server_negotiation.cc:299] Negotiation successful
0807 18:36:13.871537 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":352,"thread_start_us":134,"threads_started":1}
I20260807 18:36:13.871364 28531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:36:13.865108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.67:39057 (local address 127.0.0.1:48368)
0807 18:36:13.865989 (+ 881us) negotiation.cc:107] Waiting for socket to connect
0807 18:36:13.866003 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 18:36:13.866090 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:36:13.866955 (+ 865us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:36:13.866958 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:36:13.866974 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:36:13.867055 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:36:13.867063 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:36:13.868400 (+ 1337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:36:13.868404 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:36:13.869865 (+ 1461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:36:13.869873 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:36:13.870962 (+ 1089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:36:13.870966 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:36:13.871050 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:36:13.871082 (+ 32us) client_negotiation.cc:769] Sending connection context
0807 18:36:13.871132 (+ 50us) client_negotiation.cc:240] Negotiation successful
0807 18:36:13.871202 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":820,"mutex_wait_us":54,"thread_start_us":121,"threads_started":1}
I20260807 18:36:13.872009 28528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:36:13.865694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.66:39389 (local address 127.0.0.1:46198)
0807 18:36:13.866292 (+ 598us) negotiation.cc:107] Waiting for socket to connect
0807 18:36:13.866304 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 18:36:13.866383 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:36:13.867114 (+ 731us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:36:13.867119 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:36:13.867140 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:36:13.867218 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:36:13.867224 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:36:13.868588 (+ 1364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:36:13.868592 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:36:13.869832 (+ 1240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:36:13.869842 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:36:13.871110 (+ 1268us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:36:13.871117 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:36:13.871205 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:36:13.871224 (+ 19us) client_negotiation.cc:769] Sending connection context
0807 18:36:13.871303 (+ 79us) client_negotiation.cc:240] Negotiation successful
0807 18:36:13.871362 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":539,"mutex_wait_us":2,"thread_start_us":85,"threads_started":1}
I20260807 18:36:13.872483 28534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:36:13.866042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48368 (local address 127.11.14.67:39057)
0807 18:36:13.866501 (+ 459us) server_negotiation.cc:206] Beginning negotiation
0807 18:36:13.866520 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0807 18:36:13.866538 (+ 18us) server_negotiation.cc:407] Connection header received
0807 18:36:13.866601 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:36:13.866604 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:36:13.866793 (+ 189us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:36:13.867200 (+ 407us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:36:13.867371 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:36:13.868257 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:36:13.870005 (+ 1748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:36:13.870832 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:36:13.872040 (+ 1208us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:36:13.872203 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0807 18:36:13.872255 (+ 52us) server_negotiation.cc:299] Negotiation successful
0807 18:36:13.872346 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":354,"thread_start_us":125,"threads_started":1}
I20260807 18:36:13.874677 28142 tablet_service.cc:1467] Processing CreateTablet for tablet 7a4975cce24747bb90f390e6d22a5328 (DEFAULT_TABLE table=test-table [id=3c8e69a2661041f1b5fb9d0e6bd279cd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:36:13.874679 28287 tablet_service.cc:1467] Processing CreateTablet for tablet 7a4975cce24747bb90f390e6d22a5328 (DEFAULT_TABLE table=test-table [id=3c8e69a2661041f1b5fb9d0e6bd279cd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:36:13.875890 28142 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7a4975cce24747bb90f390e6d22a5328. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:36:13.875890 28287 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7a4975cce24747bb90f390e6d22a5328. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:36:13.878074 28438 tablet_service.cc:1467] Processing CreateTablet for tablet 7a4975cce24747bb90f390e6d22a5328 (DEFAULT_TABLE table=test-table [id=3c8e69a2661041f1b5fb9d0e6bd279cd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:36:13.879149 28438 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7a4975cce24747bb90f390e6d22a5328. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:36:13.883107 28535 tablet_bootstrap.cc:492] T 7a4975cce24747bb90f390e6d22a5328 P 8a890d9b2c9d44be9139631c5bdb2967: Bootstrap starting.
I20260807 18:36:13.884035 28536 tablet_bootstrap.cc:492] T 7a4975cce24747bb90f390e6d22a5328 P 8dd92e134d474cdfbb667f7435670c8d: Bootstrap starting.
I20260807 18:36:13.885641 28535 tablet_bootstrap.cc:654] T 7a4975cce24747bb90f390e6d22a5328 P 8a890d9b2c9d44be9139631c5bdb2967: Neither blocks nor log segments found. Creating new log.
I20260807 18:36:13.886600 28536 tablet_bootstrap.cc:654] T 7a4975cce24747bb90f390e6d22a5328 P 8dd92e134d474cdfbb667f7435670c8d: Neither blocks nor log segments found. Creating new log.
I20260807 18:36:13.886727 28535 log.cc:824] T 7a4975cce24747bb90f390e6d22a5328 P 8a890d9b2c9d44be9139631c5bdb2967: Log is configured to *not* fsync() on all Append() calls
I20260807 18:36:13.887468 28537 tablet_bootstrap.cc:492] T 7a4975cce24747bb90f390e6d22a5328 P f6c9cfee63a24a18b79f2e34c4222e44: Bootstrap starting.
I20260807 18:36:13.887458 28536 log.cc:824] T 7a4975cce24747bb90f390e6d22a5328 P 8dd92e134d474cdfbb667f7435670c8d: Log is configured to *not* fsync() on all Append() calls
I20260807 18:36:13.888788 28535 tablet_bootstrap.cc:492] T 7a4975cce24747bb90f390e6d22a5328 P 8a890d9b2c9d44be9139631c5bdb2967: No bootstrap required, opened a new log
I20260807 18:36:13.889010 28535 ts_tablet_manager.cc:1397] T 7a4975cce24747bb90f390e6d22a5328 P 8a890d9b2c9d44be9139631c5bdb2967: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260807 18:36:13.889518 28536 tablet_bootstrap.cc:492] T 7a4975cce24747bb90f390e6d22a5328 P 8dd92e134d474cdfbb667f7435670c8d: No bootstrap required, opened a new log
I20260807 18:36:13.889590 28537 tablet_bootstrap.cc:654] T 7a4975cce24747bb90f390e6d22a5328 P f6c9cfee63a24a18b79f2e34c4222e44: Neither blocks nor log segments found. Creating new log.
I20260807 18:36:13.889710 28536 ts_tablet_manager.cc:1397] T 7a4975cce24747bb90f390e6d22a5328 P 8dd92e134d474cdfbb667f7435670c8d: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.002s
I20260807 18:36:13.890784 28537 log.cc:824] T 7a4975cce24747bb90f390e6d22a5328 P f6c9cfee63a24a18b79f2e34c4222e44: Log is configured to *not* fsync() on all Append() calls
I20260807 18:36:13.892867 28537 tablet_bootstrap.cc:492] T 7a4975cce24747bb90f390e6d22a5328 P f6c9cfee63a24a18b79f2e34c4222e44: No bootstrap required, opened a new log
I20260807 18:36:13.893042 28537 ts_tablet_manager.cc:1397] T 7a4975cce24747bb90f390e6d22a5328 P f6c9cfee63a24a18b79f2e34c4222e44: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260807 18:36:13.893102 28535 raft_consensus.cc:348] T 7a4975cce24747bb90f390e6d22a5328 P 8a890d9b2c9d44be9139631c5bdb2967 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a890d9b2c9d44be9139631c5bdb2967" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 45483 } } peers { permanent_uuid: "f6c9cfee63a24a18b79f2e34c4222e44" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 39057 } } peers { permanent_uuid: "8dd92e134d474cdfbb667f7435670c8d" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 39389 } }
I20260807 18:36:13.893399 28535 raft_consensus.cc:374] T 7a4975cce24747bb90f390e6d22a5328 P 8a890d9b2c9d44be9139631c5bdb2967 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:36:13.893463 28535 raft_consensus.cc:729] T 7a4975cce24747bb90f390e6d22a5328 P 8a890d9b2c9d44be9139631c5bdb2967 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a890d9b2c9d44be9139631c5bdb2967, State: Initialized, Role: FOLLOWER
I20260807 18:36:13.893579 28536 raft_consensus.cc:348] T 7a4975cce24747bb90f390e6d22a5328 P 8dd92e134d474cdfbb667f7435670c8d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a890d9b2c9d44be9139631c5bdb2967" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 45483 } } peers { permanent_uuid: "f6c9cfee63a24a18b79f2e34c4222e44" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 39057 } } peers { permanent_uuid: "8dd92e134d474cdfbb667f7435670c8d" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 39389 } }
I20260807 18:36:13.893846 28536 raft_consensus.cc:374] T 7a4975cce24747bb90f390e6d22a5328 P 8dd92e134d474cdfbb667f7435670c8d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:36:13.893909 28536 raft_consensus.cc:729] T 7a4975cce24747bb90f390e6d22a5328 P 8dd92e134d474cdfbb667f7435670c8d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8dd92e134d474cdfbb667f7435670c8d, State: Initialized, Role: FOLLOWER
I20260807 18:36:13.893961 28535 consensus_queue.cc:260] T 7a4975cce24747bb90f390e6d22a5328 P 8a890d9b2c9d44be9139631c5bdb2967 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a890d9b2c9d44be9139631c5bdb2967" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 45483 } } peers { permanent_uuid: "f6c9cfee63a24a18b79f2e34c4222e44" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 39057 } } peers { permanent_uuid: "8dd92e134d474cdfbb667f7435670c8d" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 39389 } }
I20260807 18:36:13.894371 28536 consensus_queue.cc:260] T 7a4975cce24747bb90f390e6d22a5328 P 8dd92e134d474cdfbb667f7435670c8d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a890d9b2c9d44be9139631c5bdb2967" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 45483 } } peers { permanent_uuid: "f6c9cfee63a24a18b79f2e34c4222e44" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 39057 } } peers { permanent_uuid: "8dd92e134d474cdfbb667f7435670c8d" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 39389 } }
I20260807 18:36:13.894709 28213 heartbeater.cc:503] Master 127.11.14.126:40109 was elected leader, sending a full tablet report...
I20260807 18:36:13.895000 28535 ts_tablet_manager.cc:1428] T 7a4975cce24747bb90f390e6d22a5328 P 8a890d9b2c9d44be9139631c5bdb2967: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260807 18:36:13.895315 28536 ts_tablet_manager.cc:1428] T 7a4975cce24747bb90f390e6d22a5328 P 8dd92e134d474cdfbb667f7435670c8d: Time spent starting tablet: real 0.006s user 0.005s sys 0.001s
I20260807 18:36:13.895699 28359 heartbeater.cc:503] Master 127.11.14.126:40109 was elected leader, sending a full tablet report...
I20260807 18:36:13.896714 28537 raft_consensus.cc:348] T 7a4975cce24747bb90f390e6d22a5328 P f6c9cfee63a24a18b79f2e34c4222e44 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a890d9b2c9d44be9139631c5bdb2967" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 45483 } } peers { permanent_uuid: "f6c9cfee63a24a18b79f2e34c4222e44" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 39057 } } peers { permanent_uuid: "8dd92e134d474cdfbb667f7435670c8d" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 39389 } }
I20260807 18:36:13.897013 28537 raft_consensus.cc:374] T 7a4975cce24747bb90f390e6d22a5328 P f6c9cfee63a24a18b79f2e34c4222e44 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:36:13.897076 28537 raft_consensus.cc:729] T 7a4975cce24747bb90f390e6d22a5328 P f6c9cfee63a24a18b79f2e34c4222e44 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6c9cfee63a24a18b79f2e34c4222e44, State: Initialized, Role: FOLLOWER
I20260807 18:36:13.897511 28537 consensus_queue.cc:260] T 7a4975cce24747bb90f390e6d22a5328 P f6c9cfee63a24a18b79f2e34c4222e44 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a890d9b2c9d44be9139631c5bdb2967" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 45483 } } peers { permanent_uuid: "f6c9cfee63a24a18b79f2e34c4222e44" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 39057 } } peers { permanent_uuid: "8dd92e134d474cdfbb667f7435670c8d" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 39389 } }
I20260807 18:36:13.898434 28537 ts_tablet_manager.cc:1428] T 7a4975cce24747bb90f390e6d22a5328 P f6c9cfee63a24a18b79f2e34c4222e44: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260807 18:36:13.898813 28512 heartbeater.cc:503] Master 127.11.14.126:40109 was elected leader, sending a full tablet report...
W20260807 18:36:14.064193 28514 tablet.cc:2369] T 7a4975cce24747bb90f390e6d22a5328 P f6c9cfee63a24a18b79f2e34c4222e44: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:36:14.070197 28543 raft_consensus.cc:482] T 7a4975cce24747bb90f390e6d22a5328 P 8dd92e134d474cdfbb667f7435670c8d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:36:14.070436 28543 raft_consensus.cc:504] T 7a4975cce24747bb90f390e6d22a5328 P 8dd92e134d474cdfbb667f7435670c8d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a890d9b2c9d44be9139631c5bdb2967" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 45483 } } peers { permanent_uuid: "f6c9cfee63a24a18b79f2e34c4222e44" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 39057 } } peers { permanent_uuid: "8dd92e134d474cdfbb667f7435670c8d" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 39389 } }
I20260807 18:36:14.071723 28543 leader_election.cc:288] T 7a4975cce24747bb90f390e6d22a5328 P 8dd92e134d474cdfbb667f7435670c8d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8a890d9b2c9d44be9139631c5bdb2967 (127.11.14.65:45483), f6c9cfee63a24a18b79f2e34c4222e44 (127.11.14.67:39057)
I20260807 18:36:14.076831 28534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:36:14.072077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.66:42461 (local address 127.11.14.67:39057)
0807 18:36:14.072206 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0807 18:36:14.072212 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:36:14.072478 (+ 266us) server_negotiation.cc:407] Connection header received
0807 18:36:14.072545 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:36:14.072551 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:36:14.072603 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:36:14.072678 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:36:14.072956 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:36:14.073704 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:36:14.075498 (+ 1794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:36:14.076140 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:36:14.076282 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:36:14.076332 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0807 18:36:14.076630 (+ 298us) server_negotiation.cc:299] Negotiation successful
0807 18:36:14.076702 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260807 18:36:14.076871 28529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:36:14.072111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.66:55969 (local address 127.11.14.65:45483)
0807 18:36:14.072251 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0807 18:36:14.072258 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:36:14.072280 (+ 22us) server_negotiation.cc:407] Connection header received
0807 18:36:14.072408 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:36:14.072410 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:36:14.072481 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:36:14.072571 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:36:14.072886 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:36:14.073713 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:36:14.075502 (+ 1789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:36:14.076146 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:36:14.076278 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:36:14.076332 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0807 18:36:14.076630 (+ 298us) server_negotiation.cc:299] Negotiation successful
0807 18:36:14.076761 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260807 18:36:14.076894 28361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:36:14.071946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.65:45483 (local address 127.11.14.66:55969)
0807 18:36:14.072155 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0807 18:36:14.072170 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 18:36:14.072286 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:36:14.072597 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:36:14.072600 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:36:14.072612 (+ 12us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:36:14.072762 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:36:14.072769 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:36:14.073825 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:36:14.073828 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:36:14.075354 (+ 1526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:36:14.075366 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:36:14.076276 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:36:14.076279 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:36:14.076452 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:36:14.076517 (+ 65us) client_negotiation.cc:769] Sending connection context
0807 18:36:14.076597 (+ 80us) client_negotiation.cc:240] Negotiation successful
0807 18:36:14.076740 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"mutex_wait_us":196}
I20260807 18:36:14.077107 28545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:36:14.071999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.67:39057 (local address 127.11.14.66:42461)
0807 18:36:14.072358 (+ 359us) negotiation.cc:107] Waiting for socket to connect
0807 18:36:14.072371 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 18:36:14.072465 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:36:14.072698 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:36:14.072700 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:36:14.072716 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:36:14.072839 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:36:14.072848 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:36:14.073825 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:36:14.073828 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:36:14.075354 (+ 1526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:36:14.075366 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:36:14.076276 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:36:14.076279 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:36:14.076454 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:36:14.076516 (+ 62us) client_negotiation.cc:769] Sending connection context
0807 18:36:14.076597 (+ 81us) client_negotiation.cc:240] Negotiation successful
0807 18:36:14.076753 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":283,"mutex_wait_us":243,"thread_start_us":105,"threads_started":1}
I20260807 18:36:14.077636 28165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7a4975cce24747bb90f390e6d22a5328" candidate_uuid: "8dd92e134d474cdfbb667f7435670c8d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a890d9b2c9d44be9139631c5bdb2967" is_pre_election: true
I20260807 18:36:14.077786 28460 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7a4975cce24747bb90f390e6d22a5328" candidate_uuid: "8dd92e134d474cdfbb667f7435670c8d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6c9cfee63a24a18b79f2e34c4222e44" is_pre_election: true
I20260807 18:36:14.077970 28165 raft_consensus.cc:2454] T 7a4975cce24747bb90f390e6d22a5328 P 8a890d9b2c9d44be9139631c5bdb2967 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8dd92e134d474cdfbb667f7435670c8d in term 0.
I20260807 18:36:14.078028 28460 raft_consensus.cc:2454] T 7a4975cce24747bb90f390e6d22a5328 P f6c9cfee63a24a18b79f2e34c4222e44 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8dd92e134d474cdfbb667f7435670c8d in term 0.
I20260807 18:36:14.078553 28237 leader_election.cc:302] T 7a4975cce24747bb90f390e6d22a5328 P 8dd92e134d474cdfbb667f7435670c8d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 8a890d9b2c9d44be9139631c5bdb2967, 8dd92e134d474cdfbb667f7435670c8d, f6c9cfee63a24a18b79f2e34c4222e44; no voters:
I20260807 18:36:14.078850 28543 raft_consensus.cc:2781] T 7a4975cce24747bb90f390e6d22a5328 P 8dd92e134d474cdfbb667f7435670c8d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:36:14.078951 28543 raft_consensus.cc:482] T 7a4975cce24747bb90f390e6d22a5328 P 8dd92e134d474cdfbb667f7435670c8d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:36:14.079010 28543 raft_consensus.cc:3037] T 7a4975cce24747bb90f390e6d22a5328 P 8dd92e134d474cdfbb667f7435670c8d [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:36:14.080192 28543 raft_consensus.cc:504] T 7a4975cce24747bb90f390e6d22a5328 P 8dd92e134d474cdfbb667f7435670c8d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a890d9b2c9d44be9139631c5bdb2967" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 45483 } } peers { permanent_uuid: "f6c9cfee63a24a18b79f2e34c4222e44" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 39057 } } peers { permanent_uuid: "8dd92e134d474cdfbb667f7435670c8d" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 39389 } }
I20260807 18:36:14.080544 28543 leader_election.cc:288] T 7a4975cce24747bb90f390e6d22a5328 P 8dd92e134d474cdfbb667f7435670c8d [CANDIDATE]: Term 1 election: Requested vote from peers 8a890d9b2c9d44be9139631c5bdb2967 (127.11.14.65:45483), f6c9cfee63a24a18b79f2e34c4222e44 (127.11.14.67:39057)
I20260807 18:36:14.080919 28460 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7a4975cce24747bb90f390e6d22a5328" candidate_uuid: "8dd92e134d474cdfbb667f7435670c8d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6c9cfee63a24a18b79f2e34c4222e44"
I20260807 18:36:14.080938 28165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7a4975cce24747bb90f390e6d22a5328" candidate_uuid: "8dd92e134d474cdfbb667f7435670c8d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a890d9b2c9d44be9139631c5bdb2967"
I20260807 18:36:14.081053 28460 raft_consensus.cc:3037] T 7a4975cce24747bb90f390e6d22a5328 P f6c9cfee63a24a18b79f2e34c4222e44 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:36:14.081073 28165 raft_consensus.cc:3037] T 7a4975cce24747bb90f390e6d22a5328 P 8a890d9b2c9d44be9139631c5bdb2967 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:36:14.082072 28460 raft_consensus.cc:2454] T 7a4975cce24747bb90f390e6d22a5328 P f6c9cfee63a24a18b79f2e34c4222e44 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8dd92e134d474cdfbb667f7435670c8d in term 1.
I20260807 18:36:14.082206 28165 raft_consensus.cc:2454] T 7a4975cce24747bb90f390e6d22a5328 P 8a890d9b2c9d44be9139631c5bdb2967 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8dd92e134d474cdfbb667f7435670c8d in term 1.
I20260807 18:36:14.082438 28235 leader_election.cc:302] T 7a4975cce24747bb90f390e6d22a5328 P 8dd92e134d474cdfbb667f7435670c8d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8dd92e134d474cdfbb667f7435670c8d, f6c9cfee63a24a18b79f2e34c4222e44; no voters:
I20260807 18:36:14.082659 28543 raft_consensus.cc:2781] T 7a4975cce24747bb90f390e6d22a5328 P 8dd92e134d474cdfbb667f7435670c8d [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:36:14.082883 28543 raft_consensus.cc:686] T 7a4975cce24747bb90f390e6d22a5328 P 8dd92e134d474cdfbb667f7435670c8d [term 1 LEADER]: Becoming Leader. State: Replica: 8dd92e134d474cdfbb667f7435670c8d, State: Running, Role: LEADER
I20260807 18:36:14.083216 28543 consensus_queue.cc:237] T 7a4975cce24747bb90f390e6d22a5328 P 8dd92e134d474cdfbb667f7435670c8d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a890d9b2c9d44be9139631c5bdb2967" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 45483 } } peers { permanent_uuid: "f6c9cfee63a24a18b79f2e34c4222e44" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 39057 } } peers { permanent_uuid: "8dd92e134d474cdfbb667f7435670c8d" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 39389 } }
I20260807 18:36:14.086370 28005 catalog_manager.cc:5495] T 7a4975cce24747bb90f390e6d22a5328 P 8dd92e134d474cdfbb667f7435670c8d reported cstate change: term changed from 0 to 1, leader changed from <none> to 8dd92e134d474cdfbb667f7435670c8d (127.11.14.66). New cstate: current_term: 1 leader_uuid: "8dd92e134d474cdfbb667f7435670c8d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a890d9b2c9d44be9139631c5bdb2967" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 45483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f6c9cfee63a24a18b79f2e34c4222e44" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 39057 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8dd92e134d474cdfbb667f7435670c8d" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 39389 } health_report { overall_health: HEALTHY } } }
W20260807 18:36:14.092216 28360 tablet.cc:2369] T 7a4975cce24747bb90f390e6d22a5328 P 8dd92e134d474cdfbb667f7435670c8d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 18:36:14.102694 28214 tablet.cc:2369] T 7a4975cce24747bb90f390e6d22a5328 P 8a890d9b2c9d44be9139631c5bdb2967: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:36:14.105695 28533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:36:14.102040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46202 (local address 127.11.14.66:39389)
0807 18:36:14.102175 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0807 18:36:14.102182 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:36:14.102197 (+ 15us) server_negotiation.cc:407] Connection header received
0807 18:36:14.102401 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:36:14.102404 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:36:14.102459 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:36:14.102590 (+ 131us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:36:14.102840 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:36:14.103488 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:36:14.104358 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:36:14.104563 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:36:14.104641 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:36:14.104916 (+ 275us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:36:14.105318 (+ 402us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:36:14.105410 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0807 18:36:14.105460 (+ 50us) server_negotiation.cc:299] Negotiation successful
0807 18:36:14.105508 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260807 18:36:14.111922 28165 raft_consensus.cc:1261] T 7a4975cce24747bb90f390e6d22a5328 P 8a890d9b2c9d44be9139631c5bdb2967 [term 1 FOLLOWER]: Refusing update from remote peer 8dd92e134d474cdfbb667f7435670c8d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:36:14.111922 28460 raft_consensus.cc:1261] T 7a4975cce24747bb90f390e6d22a5328 P f6c9cfee63a24a18b79f2e34c4222e44 [term 1 FOLLOWER]: Refusing update from remote peer 8dd92e134d474cdfbb667f7435670c8d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:36:14.112648 28547 consensus_queue.cc:1035] T 7a4975cce24747bb90f390e6d22a5328 P 8dd92e134d474cdfbb667f7435670c8d [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a890d9b2c9d44be9139631c5bdb2967" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 45483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:36:14.112872 28543 consensus_queue.cc:1035] T 7a4975cce24747bb90f390e6d22a5328 P 8dd92e134d474cdfbb667f7435670c8d [LEADER]: Connected to new peer: Peer: permanent_uuid: "f6c9cfee63a24a18b79f2e34c4222e44" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 39057 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:36:14.122475 28553 mvcc.cc:205] Tried to move back new op lower bound from 7315979362754191360 to 7315979362647724032. Current Snapshot: MvccSnapshot[applied={T|T < 7315979362754191360 or (T in {7315979362754191360})}]
I20260807 18:36:14.130769 28002 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46262:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:36:14.130939 28002 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46262:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:36:14.133572 28002 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9e30e67c1f6d4fb68e8f67b10bc50f96: Sending DeleteTablet for 3 replicas of tablet 7a4975cce24747bb90f390e6d22a5328
I20260807 18:36:14.134366 28142 tablet_service.cc:1514] Processing DeleteTablet for tablet 7a4975cce24747bb90f390e6d22a5328 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:36:14 UTC) from {username='kudu', principal='kudu/127.11.14.126@KRBTEST.COM'} at 127.0.0.1:45424
I20260807 18:36:14.134516 28438 tablet_service.cc:1514] Processing DeleteTablet for tablet 7a4975cce24747bb90f390e6d22a5328 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:36:14 UTC) from {username='kudu', principal='kudu/127.11.14.126@KRBTEST.COM'} at 127.0.0.1:48368
I20260807 18:36:14.134786 28287 tablet_service.cc:1514] Processing DeleteTablet for tablet 7a4975cce24747bb90f390e6d22a5328 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:36:14 UTC) from {username='kudu', principal='kudu/127.11.14.126@KRBTEST.COM'} at 127.0.0.1:46198
I20260807 18:36:14.134965 11321 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkz56XN/build/debug/bin/kudu with pid 28075
I20260807 18:36:14.135349 28565 tablet_replica.cc:330] T 7a4975cce24747bb90f390e6d22a5328 P f6c9cfee63a24a18b79f2e34c4222e44: stopping tablet replica
I20260807 18:36:14.136797 28565 raft_consensus.cc:2229] T 7a4975cce24747bb90f390e6d22a5328 P f6c9cfee63a24a18b79f2e34c4222e44 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:36:14.137349 28565 raft_consensus.cc:2258] T 7a4975cce24747bb90f390e6d22a5328 P f6c9cfee63a24a18b79f2e34c4222e44 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:36:14.139216 28565 ts_tablet_manager.cc:1905] T 7a4975cce24747bb90f390e6d22a5328 P f6c9cfee63a24a18b79f2e34c4222e44: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:36:14.142267 28565 ts_tablet_manager.cc:1918] T 7a4975cce24747bb90f390e6d22a5328 P f6c9cfee63a24a18b79f2e34c4222e44: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:36:14.142378 28565 log.cc:1196] T 7a4975cce24747bb90f390e6d22a5328 P f6c9cfee63a24a18b79f2e34c4222e44: Deleting WAL directory at /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/minicluster-data/ts-2/wal/wals/7a4975cce24747bb90f390e6d22a5328
I20260807 18:36:14.142714 28565 ts_tablet_manager.cc:1939] T 7a4975cce24747bb90f390e6d22a5328 P f6c9cfee63a24a18b79f2e34c4222e44: Deleting consensus metadata
I20260807 18:36:14.144042 27987 catalog_manager.cc:4841] TS f6c9cfee63a24a18b79f2e34c4222e44 (127.11.14.67:39057): tablet 7a4975cce24747bb90f390e6d22a5328 (table test-table [id=3c8e69a2661041f1b5fb9d0e6bd279cd]) successfully deleted
I20260807 18:36:14.144340 28567 tablet_replica.cc:330] T 7a4975cce24747bb90f390e6d22a5328 P 8dd92e134d474cdfbb667f7435670c8d: stopping tablet replica
I20260807 18:36:14.144737 28567 raft_consensus.cc:2229] T 7a4975cce24747bb90f390e6d22a5328 P 8dd92e134d474cdfbb667f7435670c8d [term 1 LEADER]: Raft consensus shutting down.
I20260807 18:36:14.145237 28567 raft_consensus.cc:2258] T 7a4975cce24747bb90f390e6d22a5328 P 8dd92e134d474cdfbb667f7435670c8d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:36:14.146855 28567 ts_tablet_manager.cc:1905] T 7a4975cce24747bb90f390e6d22a5328 P 8dd92e134d474cdfbb667f7435670c8d: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260807 18:36:14.148105 27989 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.14.65:45483 (error 108)
I20260807 18:36:14.148370 11321 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkz56XN/build/debug/bin/kudu with pid 28218
I20260807 18:36:14.149777 28528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:36:14.149408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.65:45483 (local address 127.0.0.1:45438)
0807 18:36:14.149575 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0807 18:36:14.149659 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.65:45483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260807 18:36:14.149961 27989 catalog_manager.cc:4584] TS 8a890d9b2c9d44be9139631c5bdb2967 (127.11.14.65:45483): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 7a4975cce24747bb90f390e6d22a5328: Network error: Client connection negotiation failed: client connection to 127.11.14.65:45483: connect: Connection refused (error 111)
I20260807 18:36:14.157577 11321 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkz56XN/build/debug/bin/kudu with pid 28364
I20260807 18:36:14.157738 28528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:36:14.157425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.66:39389 (local address 127.0.0.1:46216)
0807 18:36:14.157572 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0807 18:36:14.157635 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.66:39389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260807 18:36:14.164815 11321 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkz56XN/build/debug/bin/kudu with pid 27968
I20260807 18:36:34.173213 11321 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkz56XN/build/debug/bin/kudu
/tmp/dist-test-taskkz56XN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.14.126:40109
--webserver_interface=127.11.14.126
--webserver_port=41133
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.126.keytab
--principal=kudu/127.11.14.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.14.84:35419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.14.126:40109
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:36:34.293483 28569 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:36:34.293756 28569 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:36:34.293805 28569 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:36:34.297715 28569 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260807 18:36:34.297767 28569 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260807 18:36:34.297783 28569 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260807 18:36:34.297797 28569 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:36:34.297816 28569 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:36:34.297833 28569 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:36:34.297849 28569 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:36:34.297864 28569 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:36:34.303174 28569 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.14.84:35419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.11.14.126:40109
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.126.keytab
--principal=kudu/127.11.14.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.14.126:40109
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.14.126
--webserver_port=41133
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28569
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:36:34.304221 28569 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:36:34.305223 28569 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:36:34.320158 28576 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:36:34.320269 28569 server_base.cc:1034] running on GCE node
W20260807 18:36:34.320384 28575 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:36:34.320181 28578 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:36:34.321139 28569 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:36:34.322476 28569 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:36:34.323670 28569 hybrid_clock.cc:648] HybridClock initialized: now 1786127794323639 us; error 33 us; skew 500 ppm
Aug 07 18:36:34 dist-test-slave-6k22 krb5kdc[27955](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127794, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:36:34.334607 28569 init.cc:377] Logged in from keytab as kudu/127.11.14.126@KRBTEST.COM (short username kudu)
I20260807 18:36:34.335740 28569 webserver.cc:460] Webserver started at http://127.11.14.126:41133/ using document root <none> and password file <none>
I20260807 18:36:34.336352 28569 fs_manager.cc:359] Metadata directory not provided
I20260807 18:36:34.336429 28569 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:36:34.340667 28569 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.004s
I20260807 18:36:34.342962 28587 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:36:34.344465 28569 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 18:36:34.344625 28569 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/minicluster-data/master-0/data,/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/minicluster-data/master-0/wal
uuid: "9e30e67c1f6d4fb68e8f67b10bc50f96"
format_stamp: "Formatted at 2026-08-07 18:36:13 on dist-test-slave-6k22"
server_key: "a80393781a76b348478afae6879517bd"
server_key_iv: "de5d6f34fc2b7da1a136cd32c644d014"
server_key_version: "encryptionkey@0"
I20260807 18:36:34.345291 28569 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:36:34.365670 28569 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:36:34.369125 28569 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:36:34.369378 28569 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:36:34.378189 28569 rpc_server.cc:307] RPC server started. Bound to: 127.11.14.126:40109
I20260807 18:36:34.378539 28647 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.14.126:40109 every 8 connection(s)
I20260807 18:36:34.379611 28569 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/minicluster-data/master-0/data/info.pb
I20260807 18:36:34.382431 11321 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkz56XN/build/debug/bin/kudu as pid 28569
I20260807 18:36:34.382952 11321 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkz56XN/build/debug/bin/kudu
/tmp/dist-test-taskkz56XN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.14.65:45483
--local_ip_for_outbound_sockets=127.11.14.65
--tserver_master_addrs=127.11.14.126:40109
--webserver_port=34441
--webserver_interface=127.11.14.65
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.65.keytab
--principal=kudu/127.11.14.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.14.84:35419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 18:36:34.385110 28648 sys_catalog.cc:263] Verifying existing consensus state
I20260807 18:36:34.386966 28648 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9e30e67c1f6d4fb68e8f67b10bc50f96: Bootstrap starting.
I20260807 18:36:34.405004 28648 log.cc:824] T 00000000000000000000000000000000 P 9e30e67c1f6d4fb68e8f67b10bc50f96: Log is configured to *not* fsync() on all Append() calls
I20260807 18:36:34.414063 28648 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9e30e67c1f6d4fb68e8f67b10bc50f96: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 18:36:34.414476 28648 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9e30e67c1f6d4fb68e8f67b10bc50f96: Bootstrap complete.
I20260807 18:36:34.418660 28648 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9e30e67c1f6d4fb68e8f67b10bc50f96 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e30e67c1f6d4fb68e8f67b10bc50f96" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 40109 } }
I20260807 18:36:34.419165 28648 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9e30e67c1f6d4fb68e8f67b10bc50f96 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e30e67c1f6d4fb68e8f67b10bc50f96, State: Initialized, Role: FOLLOWER
I20260807 18:36:34.419744 28648 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9e30e67c1f6d4fb68e8f67b10bc50f96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e30e67c1f6d4fb68e8f67b10bc50f96" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 40109 } }
I20260807 18:36:34.419868 28648 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9e30e67c1f6d4fb68e8f67b10bc50f96 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:36:34.419962 28648 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9e30e67c1f6d4fb68e8f67b10bc50f96 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:36:34.420076 28648 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9e30e67c1f6d4fb68e8f67b10bc50f96 [term 1 FOLLOWER]: Advancing to term 2
I20260807 18:36:34.421964 28648 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9e30e67c1f6d4fb68e8f67b10bc50f96 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e30e67c1f6d4fb68e8f67b10bc50f96" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 40109 } }
I20260807 18:36:34.422318 28648 leader_election.cc:302] T 00000000000000000000000000000000 P 9e30e67c1f6d4fb68e8f67b10bc50f96 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9e30e67c1f6d4fb68e8f67b10bc50f96; no voters:
I20260807 18:36:34.422659 28648 leader_election.cc:288] T 00000000000000000000000000000000 P 9e30e67c1f6d4fb68e8f67b10bc50f96 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260807 18:36:34.422742 28654 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9e30e67c1f6d4fb68e8f67b10bc50f96 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 18:36:34.422955 28654 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9e30e67c1f6d4fb68e8f67b10bc50f96 [term 2 LEADER]: Becoming Leader. State: Replica: 9e30e67c1f6d4fb68e8f67b10bc50f96, State: Running, Role: LEADER
I20260807 18:36:34.423251 28654 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9e30e67c1f6d4fb68e8f67b10bc50f96 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e30e67c1f6d4fb68e8f67b10bc50f96" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 40109 } }
I20260807 18:36:34.423980 28648 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9e30e67c1f6d4fb68e8f67b10bc50f96 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:36:34.424285 28654 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9e30e67c1f6d4fb68e8f67b10bc50f96 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9e30e67c1f6d4fb68e8f67b10bc50f96. Latest consensus state: current_term: 2 leader_uuid: "9e30e67c1f6d4fb68e8f67b10bc50f96" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e30e67c1f6d4fb68e8f67b10bc50f96" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 40109 } } }
I20260807 18:36:34.424422 28654 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9e30e67c1f6d4fb68e8f67b10bc50f96 [sys.catalog]: This master's current role is: LEADER
I20260807 18:36:34.424818 28655 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9e30e67c1f6d4fb68e8f67b10bc50f96 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "9e30e67c1f6d4fb68e8f67b10bc50f96" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e30e67c1f6d4fb68e8f67b10bc50f96" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 40109 } } }
I20260807 18:36:34.424989 28655 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9e30e67c1f6d4fb68e8f67b10bc50f96 [sys.catalog]: This master's current role is: LEADER
I20260807 18:36:34.426198 28658 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:36:34.431144 28658 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:36:34.431818 28658 catalog_manager.cc:1259] Loaded cluster ID: 6f64b49830594df991154b1ee8acec8f
I20260807 18:36:34.431879 28658 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:36:34.434741 28658 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:36:34.435452 28658 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 9e30e67c1f6d4fb68e8f67b10bc50f96: Loaded TSK: 0
I20260807 18:36:34.450450 28658 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9e30e67c1f6d4fb68e8f67b10bc50f96: Generated new TSK 1
I20260807 18:36:34.450749 28658 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 18:36:34.513346 28651 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:36:34.513609 28651 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:36:34.513664 28651 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:36:34.518185 28651 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:36:34.518257 28651 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:36:34.518370 28651 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.14.65
I20260807 18:36:34.523964 28651 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.14.84:35419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.65.keytab
--principal=kudu/127.11.14.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.14.65:45483
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.14.65
--webserver_port=34441
--tserver_master_addrs=127.11.14.126:40109
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28651
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.14.65
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:36:34.525144 28651 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:36:34.526289 28651 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:36:34.537801 28680 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:36:34.537844 28681 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:36:34.538121 28651 server_base.cc:1034] running on GCE node
W20260807 18:36:34.538311 28684 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:36:34.538744 28651 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:36:34.539461 28651 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:36:34.540712 28651 hybrid_clock.cc:648] HybridClock initialized: now 1786127794540674 us; error 54 us; skew 500 ppm
Aug 07 18:36:34 dist-test-slave-6k22 krb5kdc[27955](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127794, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:36:34.543766 28651 init.cc:377] Logged in from keytab as kudu/127.11.14.65@KRBTEST.COM (short username kudu)
I20260807 18:36:34.544775 28651 webserver.cc:460] Webserver started at http://127.11.14.65:34441/ using document root <none> and password file <none>
I20260807 18:36:34.545352 28651 fs_manager.cc:359] Metadata directory not provided
I20260807 18:36:34.545428 28651 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:36:34.549600 28651 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 18:36:34.552006 28691 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:36:34.553349 28651 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:36:34.553467 28651 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/minicluster-data/ts-0/data,/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/minicluster-data/ts-0/wal
uuid: "8a890d9b2c9d44be9139631c5bdb2967"
format_stamp: "Formatted at 2026-08-07 18:36:13 on dist-test-slave-6k22"
server_key: "1ff1f02b7e55fca2ea09aa675715ea75"
server_key_iv: "c3fe88fa8a4dabab522d6eec85ec86e9"
server_key_version: "encryptionkey@0"
I20260807 18:36:34.554023 28651 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:36:34.575510 28651 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:36:34.579250 28651 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:36:34.579478 28651 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:36:34.580168 28651 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:36:34.581298 28698 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260807 18:36:34.585632 28651 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260807 18:36:34.585729 28651 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.000s sys 0.000s
I20260807 18:36:34.585781 28651 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260807 18:36:34.587458 28651 ts_tablet_manager.cc:610] Registered 1 tablets
I20260807 18:36:34.587504 28651 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260807 18:36:34.587839 28698 tablet_bootstrap.cc:492] T 7a4975cce24747bb90f390e6d22a5328 P 8a890d9b2c9d44be9139631c5bdb2967: Bootstrap starting.
I20260807 18:36:34.598719 28651 rpc_server.cc:307] RPC server started. Bound to: 127.11.14.65:45483
I20260807 18:36:34.598759 28814 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.14.65:45483 every 8 connection(s)
I20260807 18:36:34.599833 28651 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/minicluster-data/ts-0/data/info.pb
I20260807 18:36:34.602370 11321 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkz56XN/build/debug/bin/kudu as pid 28651
I20260807 18:36:34.602918 11321 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkz56XN/build/debug/bin/kudu
/tmp/dist-test-taskkz56XN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.14.66:39389
--local_ip_for_outbound_sockets=127.11.14.66
--tserver_master_addrs=127.11.14.126:40109
--webserver_port=36717
--webserver_interface=127.11.14.66
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.66.keytab
--principal=kudu/127.11.14.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.14.84:35419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 18:36:34.611302 28698 log.cc:824] T 7a4975cce24747bb90f390e6d22a5328 P 8a890d9b2c9d44be9139631c5bdb2967: Log is configured to *not* fsync() on all Append() calls
Aug 07 18:36:34 dist-test-slave-6k22 krb5kdc[27955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127794, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.65@KRBTEST.COM for kudu/127.11.14.126@KRBTEST.COM
I20260807 18:36:34.620327 28818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:36:34.602134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:40109 (local address 127.11.14.65:34087)
0807 18:36:34.603474 (+ 1340us) negotiation.cc:107] Waiting for socket to connect
0807 18:36:34.603541 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0807 18:36:34.605171 (+ 1630us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:36:34.606512 (+ 1341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:36:34.606521 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:36:34.606995 (+ 474us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:36:34.607477 (+ 482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:36:34.607493 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:36:34.608911 (+ 1418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:36:34.608914 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:36:34.609919 (+ 1005us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:36:34.609929 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:36:34.610719 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:36:34.610723 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:36:34.610928 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:36:34.611918 (+ 990us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:36:34.611935 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:36:34.613651 (+ 1716us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:36:34.616528 (+ 2877us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:36:34.616538 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:36:34.616557 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:36:34.616968 (+ 411us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:36:34.617412 (+ 444us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:36:34.617418 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:36:34.617423 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:36:34.617626 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:36:34.618455 (+ 829us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:36:34.618463 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:36:34.618756 (+ 293us) client_negotiation.cc:769] Sending connection context
0807 18:36:34.618950 (+ 194us) client_negotiation.cc:240] Negotiation successful
0807 18:36:34.619299 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":769,"thread_start_us":165,"threads_started":1}
I20260807 18:36:34.620389 28817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:36:34.602111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.65:34087 (local address 127.11.14.126:40109)
0807 18:36:34.602795 (+ 684us) server_negotiation.cc:206] Beginning negotiation
0807 18:36:34.602821 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0807 18:36:34.603842 (+ 1021us) server_negotiation.cc:407] Connection header received
0807 18:36:34.605716 (+ 1874us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:36:34.605728 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:36:34.606060 (+ 332us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:36:34.606444 (+ 384us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:36:34.607728 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:36:34.608735 (+ 1007us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:36:34.610058 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:36:34.610550 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:36:34.610755 (+ 205us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:36:34.613893 (+ 3138us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:36:34.613899 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:36:34.613918 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:36:34.613954 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:36:34.616386 (+ 2432us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:36:34.617130 (+ 744us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:36:34.617137 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:36:34.617143 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:36:34.617294 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:36:34.617750 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:36:34.617753 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:36:34.617755 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:36:34.618263 (+ 508us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:36:34.618471 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0807 18:36:34.619136 (+ 665us) server_negotiation.cc:299] Negotiation successful
0807 18:36:34.619549 (+ 413us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":496,"thread_start_us":141,"threads_started":1}
I20260807 18:36:34.621053 28698 tablet_bootstrap.cc:492] T 7a4975cce24747bb90f390e6d22a5328 P 8a890d9b2c9d44be9139631c5bdb2967: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 18:36:34.621496 28698 tablet_bootstrap.cc:492] T 7a4975cce24747bb90f390e6d22a5328 P 8a890d9b2c9d44be9139631c5bdb2967: Bootstrap complete.
I20260807 18:36:34.622819 28698 ts_tablet_manager.cc:1397] T 7a4975cce24747bb90f390e6d22a5328 P 8a890d9b2c9d44be9139631c5bdb2967: Time spent bootstrapping tablet: real 0.035s user 0.021s sys 0.006s
I20260807 18:36:34.623915 28815 heartbeater.cc:347] Connected to a master server at 127.11.14.126:40109
I20260807 18:36:34.624172 28815 heartbeater.cc:464] Registering TS with master...
I20260807 18:36:34.624863 28815 heartbeater.cc:511] Master 127.11.14.126:40109 requested a full tablet report, sending...
I20260807 18:36:34.625746 28698 raft_consensus.cc:348] T 7a4975cce24747bb90f390e6d22a5328 P 8a890d9b2c9d44be9139631c5bdb2967 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a890d9b2c9d44be9139631c5bdb2967" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 45483 } } peers { permanent_uuid: "f6c9cfee63a24a18b79f2e34c4222e44" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 39057 } } peers { permanent_uuid: "8dd92e134d474cdfbb667f7435670c8d" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 39389 } }
I20260807 18:36:34.626173 28698 raft_consensus.cc:729] T 7a4975cce24747bb90f390e6d22a5328 P 8a890d9b2c9d44be9139631c5bdb2967 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a890d9b2c9d44be9139631c5bdb2967, State: Initialized, Role: FOLLOWER
I20260807 18:36:34.626708 28698 consensus_queue.cc:260] T 7a4975cce24747bb90f390e6d22a5328 P 8a890d9b2c9d44be9139631c5bdb2967 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a890d9b2c9d44be9139631c5bdb2967" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 45483 } } peers { permanent_uuid: "f6c9cfee63a24a18b79f2e34c4222e44" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 39057 } } peers { permanent_uuid: "8dd92e134d474cdfbb667f7435670c8d" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 39389 } }
I20260807 18:36:34.627795 28698 ts_tablet_manager.cc:1428] T 7a4975cce24747bb90f390e6d22a5328 P 8a890d9b2c9d44be9139631c5bdb2967: Time spent starting tablet: real 0.005s user 0.007s sys 0.001s
I20260807 18:36:34.627799 28607 ts_manager.cc:194] Registered new tserver with Master: 8a890d9b2c9d44be9139631c5bdb2967 (127.11.14.65:45483)
I20260807 18:36:34.631819 28607 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.14.65@KRBTEST.COM'} at 127.11.14.65:34087
I20260807 18:36:34.634327 28815 heartbeater.cc:503] Master 127.11.14.126:40109 was elected leader, sending a full tablet report...
Aug 07 18:36:34 dist-test-slave-6k22 krb5kdc[27955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127794, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.126@KRBTEST.COM for kudu/127.11.14.65@KRBTEST.COM
I20260807 18:36:34.642831 28824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:36:34.631239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41534 (local address 127.11.14.65:45483)
0807 18:36:34.632019 (+ 780us) server_negotiation.cc:206] Beginning negotiation
0807 18:36:34.632031 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 18:36:34.632055 (+ 24us) server_negotiation.cc:407] Connection header received
0807 18:36:34.632224 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:36:34.632232 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:36:34.632396 (+ 164us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:36:34.632546 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:36:34.633250 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:36:34.634394 (+ 1144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:36:34.635643 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:36:34.636143 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:36:34.636231 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:36:34.638817 (+ 2586us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:36:34.638827 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:36:34.638837 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:36:34.638859 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:36:34.640627 (+ 1768us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:36:34.641190 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:36:34.641201 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:36:34.641202 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:36:34.641283 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:36:34.641616 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:36:34.641620 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:36:34.641622 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:36:34.641971 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:36:34.642125 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0807 18:36:34.642490 (+ 365us) server_negotiation.cc:299] Negotiation successful
0807 18:36:34.642673 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":53,"server-negotiator.queue_time_us":684,"thread_start_us":103,"threads_started":1}
I20260807 18:36:34.642807 28823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:36:34.631180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.65:45483 (local address 127.0.0.1:41534)
0807 18:36:34.631756 (+ 576us) negotiation.cc:107] Waiting for socket to connect
0807 18:36:34.631804 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0807 18:36:34.632030 (+ 226us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:36:34.632549 (+ 519us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:36:34.632555 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:36:34.632964 (+ 409us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:36:34.633098 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:36:34.633106 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:36:34.634534 (+ 1428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:36:34.634538 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:36:34.635450 (+ 912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:36:34.635462 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:36:34.636263 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:36:34.636267 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:36:34.636373 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:36:34.636965 (+ 592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:36:34.636981 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:36:34.638651 (+ 1670us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:36:34.640770 (+ 2119us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:36:34.640781 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:36:34.640788 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:36:34.641050 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:36:34.641386 (+ 336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:36:34.641390 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:36:34.641392 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:36:34.641511 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:36:34.642098 (+ 587us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:36:34.642104 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:36:34.642249 (+ 145us) client_negotiation.cc:769] Sending connection context
0807 18:36:34.642401 (+ 152us) client_negotiation.cc:240] Negotiation successful
0807 18:36:34.642621 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":356,"thread_start_us":129,"threads_started":1}
I20260807 18:36:34.643954 28744 tablet_service.cc:1514] Processing DeleteTablet for tablet 7a4975cce24747bb90f390e6d22a5328 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:36:14 UTC) from {username='kudu', principal='kudu/127.11.14.126@KRBTEST.COM'} at 127.0.0.1:41534
I20260807 18:36:34.644912 28825 tablet_replica.cc:330] stopping tablet replica
I20260807 18:36:34.645299 28825 raft_consensus.cc:2229] T 7a4975cce24747bb90f390e6d22a5328 P 8a890d9b2c9d44be9139631c5bdb2967 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:36:34.645608 28825 raft_consensus.cc:2258] T 7a4975cce24747bb90f390e6d22a5328 P 8a890d9b2c9d44be9139631c5bdb2967 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:36:34.646800 28825 ts_tablet_manager.cc:1905] T 7a4975cce24747bb90f390e6d22a5328 P 8a890d9b2c9d44be9139631c5bdb2967: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:36:34.651813 28825 ts_tablet_manager.cc:1918] T 7a4975cce24747bb90f390e6d22a5328 P 8a890d9b2c9d44be9139631c5bdb2967: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:36:34.651940 28825 log.cc:1196] T 7a4975cce24747bb90f390e6d22a5328 P 8a890d9b2c9d44be9139631c5bdb2967: Deleting WAL directory at /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/minicluster-data/ts-0/wal/wals/7a4975cce24747bb90f390e6d22a5328
I20260807 18:36:34.652271 28825 ts_tablet_manager.cc:1939] T 7a4975cce24747bb90f390e6d22a5328 P 8a890d9b2c9d44be9139631c5bdb2967: Deleting consensus metadata
I20260807 18:36:34.654072 28590 catalog_manager.cc:4841] TS 8a890d9b2c9d44be9139631c5bdb2967 (127.11.14.65:45483): tablet 7a4975cce24747bb90f390e6d22a5328 (table test-table [id=3c8e69a2661041f1b5fb9d0e6bd279cd]) successfully deleted
W20260807 18:36:34.733291 28819 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:36:34.733531 28819 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:36:34.733573 28819 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:36:34.737792 28819 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:36:34.737855 28819 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:36:34.737915 28819 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.14.66
I20260807 18:36:34.743366 28819 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.14.84:35419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.66.keytab
--principal=kudu/127.11.14.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.14.66:39389
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.14.66
--webserver_port=36717
--tserver_master_addrs=127.11.14.126:40109
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28819
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.14.66
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:36:34.744552 28819 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:36:34.745611 28819 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:36:34.755643 28831 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:36:34.755766 28819 server_base.cc:1034] running on GCE node
W20260807 18:36:34.755704 28833 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:36:34.755709 28830 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:36:34.756366 28819 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:36:34.757140 28819 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:36:34.758509 28819 hybrid_clock.cc:648] HybridClock initialized: now 1786127794758478 us; error 34 us; skew 500 ppm
Aug 07 18:36:34 dist-test-slave-6k22 krb5kdc[27955](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127794, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:36:34.761478 28819 init.cc:377] Logged in from keytab as kudu/127.11.14.66@KRBTEST.COM (short username kudu)
I20260807 18:36:34.762341 28819 webserver.cc:460] Webserver started at http://127.11.14.66:36717/ using document root <none> and password file <none>
I20260807 18:36:34.762884 28819 fs_manager.cc:359] Metadata directory not provided
I20260807 18:36:34.762960 28819 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:36:34.766880 28819 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260807 18:36:34.769127 28841 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:36:34.770428 28819 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 18:36:34.770527 28819 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/minicluster-data/ts-1/data,/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/minicluster-data/ts-1/wal
uuid: "8dd92e134d474cdfbb667f7435670c8d"
format_stamp: "Formatted at 2026-08-07 18:36:13 on dist-test-slave-6k22"
server_key: "f57f065283ab1a4e5f3b246de1834c43"
server_key_iv: "0653dbc1ced63c01d8f6843d603fe578"
server_key_version: "encryptionkey@0"
I20260807 18:36:34.771045 28819 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:36:34.794915 28819 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:36:34.798190 28819 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:36:34.798388 28819 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:36:34.798996 28819 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:36:34.800246 28848 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
W20260807 18:36:34.804221 28848 ts_tablet_manager.cc:1852] T 7a4975cce24747bb90f390e6d22a5328 P 8dd92e134d474cdfbb667f7435670c8d: Tablet Manager startup: Rolling forward tablet deletion of type TABLET_DATA_DELETED
I20260807 18:36:34.804613 28848 ts_tablet_manager.cc:1905] T 7a4975cce24747bb90f390e6d22a5328 P 8dd92e134d474cdfbb667f7435670c8d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:36:34.808630 28848 ts_tablet_manager.cc:1918] T 7a4975cce24747bb90f390e6d22a5328 P 8dd92e134d474cdfbb667f7435670c8d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:36:34.808729 28848 log.cc:1196] T 7a4975cce24747bb90f390e6d22a5328 P 8dd92e134d474cdfbb667f7435670c8d: Deleting WAL directory at /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/minicluster-data/ts-1/wal/wals/7a4975cce24747bb90f390e6d22a5328
I20260807 18:36:34.809113 28848 ts_tablet_manager.cc:1939] T 7a4975cce24747bb90f390e6d22a5328 P 8dd92e134d474cdfbb667f7435670c8d: Deleting consensus metadata
I20260807 18:36:34.809456 28819 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 0 live tablets)
I20260807 18:36:34.809543 28819 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.010s user 0.000s sys 0.001s
I20260807 18:36:34.809667 28819 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:36:34.809711 28819 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:36:34.820254 28819 rpc_server.cc:307] RPC server started. Bound to: 127.11.14.66:39389
I20260807 18:36:34.820290 28966 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.14.66:39389 every 8 connection(s)
I20260807 18:36:34.821274 28819 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/minicluster-data/ts-1/data/info.pb
I20260807 18:36:34.822836 11321 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkz56XN/build/debug/bin/kudu as pid 28819
I20260807 18:36:34.823295 11321 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkz56XN/build/debug/bin/kudu
/tmp/dist-test-taskkz56XN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.14.67:39057
--local_ip_for_outbound_sockets=127.11.14.67
--tserver_master_addrs=127.11.14.126:40109
--webserver_port=38235
--webserver_interface=127.11.14.67
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.67.keytab
--principal=kudu/127.11.14.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.14.84:35419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 18:36:34 dist-test-slave-6k22 krb5kdc[27955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127794, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.66@KRBTEST.COM for kudu/127.11.14.126@KRBTEST.COM
I20260807 18:36:34.836870 28817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:36:34.823472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.66:60551 (local address 127.11.14.126:40109)
0807 18:36:34.823671 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0807 18:36:34.823677 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:36:34.824948 (+ 1271us) server_negotiation.cc:407] Connection header received
0807 18:36:34.826209 (+ 1261us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:36:34.826213 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:36:34.826283 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:36:34.826398 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:36:34.827710 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:36:34.828318 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:36:34.829599 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:36:34.829873 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:36:34.830006 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:36:34.832796 (+ 2790us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:36:34.832803 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:36:34.832806 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:36:34.832839 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:36:34.834448 (+ 1609us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:36:34.835027 (+ 579us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:36:34.835031 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:36:34.835034 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:36:34.835126 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:36:34.835590 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:36:34.835594 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:36:34.835596 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:36:34.835818 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:36:34.835943 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0807 18:36:34.836509 (+ 566us) server_negotiation.cc:299] Negotiation successful
0807 18:36:34.836674 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260807 18:36:34.837761 28970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:36:34.824072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:40109 (local address 127.11.14.66:60551)
0807 18:36:34.824636 (+ 564us) negotiation.cc:107] Waiting for socket to connect
0807 18:36:34.824802 (+ 166us) client_negotiation.cc:174] Beginning negotiation
0807 18:36:34.825919 (+ 1117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:36:34.826594 (+ 675us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:36:34.826602 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:36:34.826976 (+ 374us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:36:34.827489 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:36:34.827503 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:36:34.828470 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:36:34.828474 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:36:34.829439 (+ 965us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:36:34.829449 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:36:34.830052 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:36:34.830057 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:36:34.830250 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:36:34.830953 (+ 703us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:36:34.830971 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:36:34.832595 (+ 1624us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:36:34.834587 (+ 1992us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:36:34.834593 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:36:34.834610 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:36:34.834900 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:36:34.835280 (+ 380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:36:34.835287 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:36:34.835289 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:36:34.835447 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:36:34.835955 (+ 508us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:36:34.835965 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:36:34.836235 (+ 270us) client_negotiation.cc:769] Sending connection context
0807 18:36:34.836448 (+ 213us) client_negotiation.cc:240] Negotiation successful
0807 18:36:34.836838 (+ 390us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":327,"thread_start_us":187,"threads_started":1}
I20260807 18:36:34.839247 28968 heartbeater.cc:347] Connected to a master server at 127.11.14.126:40109
I20260807 18:36:34.839519 28968 heartbeater.cc:464] Registering TS with master...
I20260807 18:36:34.840138 28968 heartbeater.cc:511] Master 127.11.14.126:40109 requested a full tablet report, sending...
I20260807 18:36:34.841428 28607 ts_manager.cc:194] Registered new tserver with Master: 8dd92e134d474cdfbb667f7435670c8d (127.11.14.66:39389)
I20260807 18:36:34.842043 28607 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.14.66@KRBTEST.COM'} at 127.11.14.66:60551
W20260807 18:36:34.942133 28971 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:36:34.942430 28971 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:36:34.942513 28971 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:36:34.946656 28971 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:36:34.946749 28971 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:36:34.946841 28971 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.14.67
I20260807 18:36:34.952539 28971 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.14.84:35419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.67.keytab
--principal=kudu/127.11.14.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.14.67:39057
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.14.67
--webserver_port=38235
--tserver_master_addrs=127.11.14.126:40109
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28971
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.14.67
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:36:34.953603 28971 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:36:34.954623 28971 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:36:34.964668 28980 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:36:34.964627 28978 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:36:34.964660 28977 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:36:34.964937 28971 server_base.cc:1034] running on GCE node
I20260807 18:36:34.965440 28971 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:36:34.966099 28971 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:36:34.967279 28971 hybrid_clock.cc:648] HybridClock initialized: now 1786127794967260 us; error 27 us; skew 500 ppm
Aug 07 18:36:34 dist-test-slave-6k22 krb5kdc[27955](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127794, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:36:34.970511 28971 init.cc:377] Logged in from keytab as kudu/127.11.14.67@KRBTEST.COM (short username kudu)
I20260807 18:36:34.971462 28971 webserver.cc:460] Webserver started at http://127.11.14.67:38235/ using document root <none> and password file <none>
I20260807 18:36:34.972069 28971 fs_manager.cc:359] Metadata directory not provided
I20260807 18:36:34.972157 28971 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:36:34.976341 28971 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 18:36:34.978564 28988 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:36:34.979972 28971 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:36:34.980082 28971 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/minicluster-data/ts-2/data,/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/minicluster-data/ts-2/wal
uuid: "f6c9cfee63a24a18b79f2e34c4222e44"
format_stamp: "Formatted at 2026-08-07 18:36:13 on dist-test-slave-6k22"
server_key: "1f168783971fc54e3f339eabb2b63d2b"
server_key_iv: "9d915462a34803bbef932b44281d251e"
server_key_version: "encryptionkey@0"
I20260807 18:36:34.980629 28971 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:36:34.998340 28971 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:36:35.001647 28971 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:36:35.001829 28971 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:36:35.002416 28971 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:36:35.003305 28971 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:36:35.003357 28971 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:36:35.003401 28971 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:36:35.003424 28971 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:36:35.013438 28971 rpc_server.cc:307] RPC server started. Bound to: 127.11.14.67:39057
I20260807 18:36:35.013474 29113 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.14.67:39057 every 8 connection(s)
I20260807 18:36:35.014539 28971 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/minicluster-data/ts-2/data/info.pb
I20260807 18:36:35.021687 11321 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkz56XN/build/debug/bin/kudu as pid 28971
Aug 07 18:36:35 dist-test-slave-6k22 krb5kdc[27955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127794, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.67@KRBTEST.COM for kudu/127.11.14.126@KRBTEST.COM
I20260807 18:36:35.029717 28817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:36:35.016568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.67:54755 (local address 127.11.14.126:40109)
0807 18:36:35.016776 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0807 18:36:35.016780 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:36:35.018007 (+ 1227us) server_negotiation.cc:407] Connection header received
0807 18:36:35.018877 (+ 870us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:36:35.018880 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:36:35.018925 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:36:35.019002 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:36:35.020314 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:36:35.020824 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:36:35.022035 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:36:35.022308 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:36:35.022415 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:36:35.025594 (+ 3179us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:36:35.025597 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:36:35.025599 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:36:35.025673 (+ 74us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:36:35.027456 (+ 1783us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:36:35.028096 (+ 640us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:36:35.028099 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:36:35.028100 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:36:35.028185 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:36:35.028583 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:36:35.028586 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:36:35.028587 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:36:35.028766 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:36:35.028882 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0807 18:36:35.029404 (+ 522us) server_negotiation.cc:299] Negotiation successful
0807 18:36:35.029529 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260807 18:36:35.030498 29116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:36:35.017083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:40109 (local address 127.11.14.67:54755)
0807 18:36:35.017825 (+ 742us) negotiation.cc:107] Waiting for socket to connect
0807 18:36:35.017870 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 18:36:35.018681 (+ 811us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:36:35.019184 (+ 503us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:36:35.019191 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:36:35.019680 (+ 489us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:36:35.020181 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:36:35.020195 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:36:35.020941 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:36:35.020944 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:36:35.021902 (+ 958us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:36:35.021917 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:36:35.022593 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:36:35.022597 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:36:35.022812 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:36:35.023529 (+ 717us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:36:35.023545 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:36:35.025423 (+ 1878us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:36:35.027622 (+ 2199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:36:35.027630 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:36:35.027647 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:36:35.027989 (+ 342us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:36:35.028297 (+ 308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:36:35.028301 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:36:35.028303 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:36:35.028454 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:36:35.028883 (+ 429us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:36:35.028891 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:36:35.029139 (+ 248us) client_negotiation.cc:769] Sending connection context
0807 18:36:35.029339 (+ 200us) client_negotiation.cc:240] Negotiation successful
0807 18:36:35.029664 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":480,"thread_start_us":115,"threads_started":1}
I20260807 18:36:35.031419 29117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:36:35.022437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50118 (local address 127.11.14.126:40109)
0807 18:36:35.022776 (+ 339us) server_negotiation.cc:206] Beginning negotiation
0807 18:36:35.022780 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:36:35.022902 (+ 122us) server_negotiation.cc:407] Connection header received
0807 18:36:35.023008 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:36:35.023011 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:36:35.023067 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:36:35.023173 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:36:35.024084 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:36:35.024755 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:36:35.025778 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:36:35.026113 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:36:35.026222 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:36:35.027904 (+ 1682us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:36:35.027908 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:36:35.027910 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:36:35.027937 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:36:35.029442 (+ 1505us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:36:35.029938 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:36:35.029941 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:36:35.029942 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:36:35.030021 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:36:35.030294 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:36:35.030298 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:36:35.030299 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:36:35.030468 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:36:35.030684 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0807 18:36:35.031119 (+ 435us) server_negotiation.cc:299] Negotiation successful
0807 18:36:35.031252 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":267,"thread_start_us":103,"threads_started":1}
I20260807 18:36:35.031911 29114 heartbeater.cc:347] Connected to a master server at 127.11.14.126:40109
I20260807 18:36:35.032183 29114 heartbeater.cc:464] Registering TS with master...
I20260807 18:36:35.032769 29114 heartbeater.cc:511] Master 127.11.14.126:40109 requested a full tablet report, sending...
I20260807 18:36:35.033797 28607 ts_manager.cc:194] Registered new tserver with Master: f6c9cfee63a24a18b79f2e34c4222e44 (127.11.14.67:39057)
I20260807 18:36:35.034353 28607 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.14.67@KRBTEST.COM'} at 127.11.14.67:54755
I20260807 18:36:35.045279 11321 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 18:36:35.049494 29117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:36:35.045939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50122 (local address 127.11.14.126:40109)
0807 18:36:35.046067 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0807 18:36:35.046070 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 18:36:35.046320 (+ 250us) server_negotiation.cc:407] Connection header received
0807 18:36:35.046434 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:36:35.046436 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:36:35.046545 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:36:35.046671 (+ 126us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:36:35.046928 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:36:35.047467 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:36:35.048393 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:36:35.048613 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:36:35.048694 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:36:35.049010 (+ 316us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:36:35.049180 (+ 170us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0807 18:36:35.049394 (+ 214us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:50122: token expired
Metrics: {"server-negotiator.queue_time_us":52}
W20260807 18:36:35.049574 29117 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:50122: token expired
W20260807 18:36:35.049929 28518 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.11.14.126:40109)
I20260807 18:36:35.057310 29117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:36:35.050355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50130 (local address 127.11.14.126:40109)
0807 18:36:35.050464 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0807 18:36:35.050466 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 18:36:35.050479 (+ 13us) server_negotiation.cc:407] Connection header received
0807 18:36:35.050538 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:36:35.050540 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:36:35.050570 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:36:35.050632 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:36:35.051244 (+ 612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:36:35.051753 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:36:35.052583 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:36:35.052788 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:36:35.052856 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:36:35.054307 (+ 1451us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:36:35.054310 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:36:35.054312 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:36:35.054333 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:36:35.055848 (+ 1515us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:36:35.056262 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:36:35.056264 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:36:35.056265 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:36:35.056336 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:36:35.056563 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:36:35.056565 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:36:35.056566 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:36:35.056717 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:36:35.056828 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0807 18:36:35.057022 (+ 194us) server_negotiation.cc:299] Negotiation successful
0807 18:36:35.057197 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260807 18:36:35.059963 28607 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50130:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 18:36:35.063208 28607 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 18:36:35.071460 28744 tablet_service.cc:1467] Processing CreateTablet for tablet df5fba67196f4c6892658b1580fdc44d (DEFAULT_TABLE table=test-table [id=fc006f47ca0741f48209f016cd610f7d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:36:35.072211 28744 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet df5fba67196f4c6892658b1580fdc44d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:36:35.075122 28698 tablet_bootstrap.cc:492] T df5fba67196f4c6892658b1580fdc44d P 8a890d9b2c9d44be9139631c5bdb2967: Bootstrap starting.
I20260807 18:36:35.076061 28698 tablet_bootstrap.cc:654] T df5fba67196f4c6892658b1580fdc44d P 8a890d9b2c9d44be9139631c5bdb2967: Neither blocks nor log segments found. Creating new log.
I20260807 18:36:35.076674 29126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:36:35.071011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34904 (local address 127.11.14.66:39389)
0807 18:36:35.071416 (+ 405us) server_negotiation.cc:206] Beginning negotiation
0807 18:36:35.071425 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:36:35.071444 (+ 19us) server_negotiation.cc:407] Connection header received
0807 18:36:35.071508 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:36:35.071511 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:36:35.071780 (+ 269us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:36:35.072053 (+ 273us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:36:35.072464 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:36:35.073424 (+ 960us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:36:35.075005 (+ 1581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:36:35.075917 (+ 912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:36:35.076234 (+ 317us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:36:35.076385 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0807 18:36:35.076462 (+ 77us) server_negotiation.cc:299] Negotiation successful
0807 18:36:35.076544 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":321,"thread_start_us":109,"threads_started":1}
I20260807 18:36:35.076689 29125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:36:35.070719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.66:39389 (local address 127.0.0.1:34904)
0807 18:36:35.071035 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0807 18:36:35.071049 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 18:36:35.071171 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:36:35.071938 (+ 767us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:36:35.071941 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:36:35.071973 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:36:35.072334 (+ 361us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:36:35.072343 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:36:35.073551 (+ 1208us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:36:35.073556 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:36:35.074856 (+ 1300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:36:35.074867 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:36:35.076057 (+ 1190us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:36:35.076061 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:36:35.076265 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:36:35.076322 (+ 57us) client_negotiation.cc:769] Sending connection context
0807 18:36:35.076405 (+ 83us) client_negotiation.cc:240] Negotiation successful
0807 18:36:35.076531 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":260,"mutex_wait_us":284,"spinlock_wait_cycles":8064,"thread_start_us":107,"threads_started":1}
I20260807 18:36:35.077242 28823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:36:35.070337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.67:39057 (local address 127.0.0.1:37832)
0807 18:36:35.070482 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0807 18:36:35.070500 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 18:36:35.070588 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:36:35.071938 (+ 1350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:36:35.071941 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:36:35.071971 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:36:35.072114 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:36:35.072124 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:36:35.073821 (+ 1697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:36:35.073825 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:36:35.074962 (+ 1137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:36:35.074973 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:36:35.076057 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:36:35.076061 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:36:35.076207 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:36:35.076971 (+ 764us) client_negotiation.cc:769] Sending connection context
0807 18:36:35.077042 (+ 71us) client_negotiation.cc:240] Negotiation successful
0807 18:36:35.077122 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":63,"mutex_wait_us":742,"spinlock_wait_cycles":5120}
I20260807 18:36:35.077315 29124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:36:35.070557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37832 (local address 127.11.14.67:39057)
0807 18:36:35.071212 (+ 655us) server_negotiation.cc:206] Beginning negotiation
0807 18:36:35.071225 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 18:36:35.071259 (+ 34us) server_negotiation.cc:407] Connection header received
0807 18:36:35.071341 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:36:35.071346 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:36:35.071610 (+ 264us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:36:35.071961 (+ 351us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:36:35.072607 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:36:35.073702 (+ 1095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:36:35.075088 (+ 1386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:36:35.075917 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:36:35.076261 (+ 344us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:36:35.076385 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0807 18:36:35.077079 (+ 694us) server_negotiation.cc:299] Negotiation successful
0807 18:36:35.077176 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":551,"thread_start_us":171,"threads_started":1}
I20260807 18:36:35.078307 28698 tablet_bootstrap.cc:492] T df5fba67196f4c6892658b1580fdc44d P 8a890d9b2c9d44be9139631c5bdb2967: No bootstrap required, opened a new log
I20260807 18:36:35.078418 28698 ts_tablet_manager.cc:1397] T df5fba67196f4c6892658b1580fdc44d P 8a890d9b2c9d44be9139631c5bdb2967: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260807 18:36:35.078306 28894 tablet_service.cc:1467] Processing CreateTablet for tablet df5fba67196f4c6892658b1580fdc44d (DEFAULT_TABLE table=test-table [id=fc006f47ca0741f48209f016cd610f7d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:36:35.078861 28698 raft_consensus.cc:348] T df5fba67196f4c6892658b1580fdc44d P 8a890d9b2c9d44be9139631c5bdb2967 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6c9cfee63a24a18b79f2e34c4222e44" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 39057 } } peers { permanent_uuid: "8dd92e134d474cdfbb667f7435670c8d" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 39389 } } peers { permanent_uuid: "8a890d9b2c9d44be9139631c5bdb2967" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 45483 } }
I20260807 18:36:35.078972 28698 raft_consensus.cc:374] T df5fba67196f4c6892658b1580fdc44d P 8a890d9b2c9d44be9139631c5bdb2967 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:36:35.079008 28698 raft_consensus.cc:729] T df5fba67196f4c6892658b1580fdc44d P 8a890d9b2c9d44be9139631c5bdb2967 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a890d9b2c9d44be9139631c5bdb2967, State: Initialized, Role: FOLLOWER
I20260807 18:36:35.079137 28698 consensus_queue.cc:260] T df5fba67196f4c6892658b1580fdc44d P 8a890d9b2c9d44be9139631c5bdb2967 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6c9cfee63a24a18b79f2e34c4222e44" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 39057 } } peers { permanent_uuid: "8dd92e134d474cdfbb667f7435670c8d" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 39389 } } peers { permanent_uuid: "8a890d9b2c9d44be9139631c5bdb2967" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 45483 } }
I20260807 18:36:35.079397 28698 ts_tablet_manager.cc:1428] T df5fba67196f4c6892658b1580fdc44d P 8a890d9b2c9d44be9139631c5bdb2967: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260807 18:36:35.079486 28894 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet df5fba67196f4c6892658b1580fdc44d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:36:35.080492 29041 tablet_service.cc:1467] Processing CreateTablet for tablet df5fba67196f4c6892658b1580fdc44d (DEFAULT_TABLE table=test-table [id=fc006f47ca0741f48209f016cd610f7d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:36:35.081452 29041 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet df5fba67196f4c6892658b1580fdc44d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:36:35.084479 28848 tablet_bootstrap.cc:492] T df5fba67196f4c6892658b1580fdc44d P 8dd92e134d474cdfbb667f7435670c8d: Bootstrap starting.
I20260807 18:36:35.086807 28848 tablet_bootstrap.cc:654] T df5fba67196f4c6892658b1580fdc44d P 8dd92e134d474cdfbb667f7435670c8d: Neither blocks nor log segments found. Creating new log.
I20260807 18:36:35.087776 28848 log.cc:824] T df5fba67196f4c6892658b1580fdc44d P 8dd92e134d474cdfbb667f7435670c8d: Log is configured to *not* fsync() on all Append() calls
I20260807 18:36:35.089319 29129 tablet_bootstrap.cc:492] T df5fba67196f4c6892658b1580fdc44d P f6c9cfee63a24a18b79f2e34c4222e44: Bootstrap starting.
I20260807 18:36:35.089937 28848 tablet_bootstrap.cc:492] T df5fba67196f4c6892658b1580fdc44d P 8dd92e134d474cdfbb667f7435670c8d: No bootstrap required, opened a new log
I20260807 18:36:35.090127 28848 ts_tablet_manager.cc:1397] T df5fba67196f4c6892658b1580fdc44d P 8dd92e134d474cdfbb667f7435670c8d: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260807 18:36:35.091398 29129 tablet_bootstrap.cc:654] T df5fba67196f4c6892658b1580fdc44d P f6c9cfee63a24a18b79f2e34c4222e44: Neither blocks nor log segments found. Creating new log.
I20260807 18:36:35.092585 29129 log.cc:824] T df5fba67196f4c6892658b1580fdc44d P f6c9cfee63a24a18b79f2e34c4222e44: Log is configured to *not* fsync() on all Append() calls
I20260807 18:36:35.093310 28848 raft_consensus.cc:348] T df5fba67196f4c6892658b1580fdc44d P 8dd92e134d474cdfbb667f7435670c8d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6c9cfee63a24a18b79f2e34c4222e44" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 39057 } } peers { permanent_uuid: "8dd92e134d474cdfbb667f7435670c8d" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 39389 } } peers { permanent_uuid: "8a890d9b2c9d44be9139631c5bdb2967" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 45483 } }
I20260807 18:36:35.093555 28848 raft_consensus.cc:374] T df5fba67196f4c6892658b1580fdc44d P 8dd92e134d474cdfbb667f7435670c8d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:36:35.093614 28848 raft_consensus.cc:729] T df5fba67196f4c6892658b1580fdc44d P 8dd92e134d474cdfbb667f7435670c8d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8dd92e134d474cdfbb667f7435670c8d, State: Initialized, Role: FOLLOWER
I20260807 18:36:35.094082 28848 consensus_queue.cc:260] T df5fba67196f4c6892658b1580fdc44d P 8dd92e134d474cdfbb667f7435670c8d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6c9cfee63a24a18b79f2e34c4222e44" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 39057 } } peers { permanent_uuid: "8dd92e134d474cdfbb667f7435670c8d" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 39389 } } peers { permanent_uuid: "8a890d9b2c9d44be9139631c5bdb2967" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 45483 } }
I20260807 18:36:35.094539 29129 tablet_bootstrap.cc:492] T df5fba67196f4c6892658b1580fdc44d P f6c9cfee63a24a18b79f2e34c4222e44: No bootstrap required, opened a new log
I20260807 18:36:35.094729 29129 ts_tablet_manager.cc:1397] T df5fba67196f4c6892658b1580fdc44d P f6c9cfee63a24a18b79f2e34c4222e44: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260807 18:36:35.094763 28968 heartbeater.cc:503] Master 127.11.14.126:40109 was elected leader, sending a full tablet report...
I20260807 18:36:35.095032 28848 ts_tablet_manager.cc:1428] T df5fba67196f4c6892658b1580fdc44d P 8dd92e134d474cdfbb667f7435670c8d: Time spent starting tablet: real 0.005s user 0.001s sys 0.005s
I20260807 18:36:35.097921 29129 raft_consensus.cc:348] T df5fba67196f4c6892658b1580fdc44d P f6c9cfee63a24a18b79f2e34c4222e44 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6c9cfee63a24a18b79f2e34c4222e44" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 39057 } } peers { permanent_uuid: "8dd92e134d474cdfbb667f7435670c8d" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 39389 } } peers { permanent_uuid: "8a890d9b2c9d44be9139631c5bdb2967" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 45483 } }
I20260807 18:36:35.098153 29129 raft_consensus.cc:374] T df5fba67196f4c6892658b1580fdc44d P f6c9cfee63a24a18b79f2e34c4222e44 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:36:35.098219 29129 raft_consensus.cc:729] T df5fba67196f4c6892658b1580fdc44d P f6c9cfee63a24a18b79f2e34c4222e44 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6c9cfee63a24a18b79f2e34c4222e44, State: Initialized, Role: FOLLOWER
I20260807 18:36:35.098657 29129 consensus_queue.cc:260] T df5fba67196f4c6892658b1580fdc44d P f6c9cfee63a24a18b79f2e34c4222e44 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6c9cfee63a24a18b79f2e34c4222e44" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 39057 } } peers { permanent_uuid: "8dd92e134d474cdfbb667f7435670c8d" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 39389 } } peers { permanent_uuid: "8a890d9b2c9d44be9139631c5bdb2967" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 45483 } }
I20260807 18:36:35.099326 29114 heartbeater.cc:503] Master 127.11.14.126:40109 was elected leader, sending a full tablet report...
I20260807 18:36:35.099746 29129 ts_tablet_manager.cc:1428] T df5fba67196f4c6892658b1580fdc44d P f6c9cfee63a24a18b79f2e34c4222e44: Time spent starting tablet: real 0.005s user 0.006s sys 0.001s
W20260807 18:36:35.101190 28816 tablet.cc:2369] T df5fba67196f4c6892658b1580fdc44d P 8a890d9b2c9d44be9139631c5bdb2967: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:36:35.173040 29132 raft_consensus.cc:482] T df5fba67196f4c6892658b1580fdc44d P f6c9cfee63a24a18b79f2e34c4222e44 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:36:35.173301 29132 raft_consensus.cc:504] T df5fba67196f4c6892658b1580fdc44d P f6c9cfee63a24a18b79f2e34c4222e44 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6c9cfee63a24a18b79f2e34c4222e44" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 39057 } } peers { permanent_uuid: "8dd92e134d474cdfbb667f7435670c8d" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 39389 } } peers { permanent_uuid: "8a890d9b2c9d44be9139631c5bdb2967" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 45483 } }
I20260807 18:36:35.174836 29132 leader_election.cc:288] T df5fba67196f4c6892658b1580fdc44d P f6c9cfee63a24a18b79f2e34c4222e44 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8dd92e134d474cdfbb667f7435670c8d (127.11.14.66:39389), 8a890d9b2c9d44be9139631c5bdb2967 (127.11.14.65:45483)
I20260807 18:36:35.179334 29116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:36:35.175062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.66:39389 (local address 127.11.14.67:39405)
0807 18:36:35.175242 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0807 18:36:35.175273 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0807 18:36:35.175371 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:36:35.175794 (+ 423us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:36:35.175798 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:36:35.175828 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:36:35.176012 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:36:35.176022 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:36:35.176964 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:36:35.176967 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:36:35.177958 (+ 991us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:36:35.177969 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:36:35.178890 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:36:35.178896 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:36:35.179014 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:36:35.179037 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 18:36:35.179114 (+ 77us) client_negotiation.cc:240] Negotiation successful
0807 18:36:35.179192 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":90,"mutex_wait_us":26}
I20260807 18:36:35.179364 29126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:36:35.175308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.67:39405 (local address 127.11.14.66:39389)
0807 18:36:35.175472 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0807 18:36:35.175476 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:36:35.175493 (+ 17us) server_negotiation.cc:407] Connection header received
0807 18:36:35.175552 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:36:35.175554 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:36:35.175639 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:36:35.175730 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:36:35.176186 (+ 456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:36:35.176824 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:36:35.178099 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:36:35.178731 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:36:35.178854 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:36:35.178899 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0807 18:36:35.179153 (+ 254us) server_negotiation.cc:299] Negotiation successful
0807 18:36:35.179237 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260807 18:36:35.179415 29134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:36:35.175277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.67:45353 (local address 127.11.14.65:45483)
0807 18:36:35.175613 (+ 336us) server_negotiation.cc:206] Beginning negotiation
0807 18:36:35.175622 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:36:35.175790 (+ 168us) server_negotiation.cc:407] Connection header received
0807 18:36:35.175933 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:36:35.175937 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:36:35.176018 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:36:35.176179 (+ 161us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:36:35.176502 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:36:35.177194 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:36:35.178362 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:36:35.178855 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:36:35.178985 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:36:35.179167 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0807 18:36:35.179208 (+ 41us) server_negotiation.cc:299] Negotiation successful
0807 18:36:35.179281 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":211,"thread_start_us":87,"threads_started":1}
I20260807 18:36:35.179792 29133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:36:35.175184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.65:45483 (local address 127.11.14.67:45353)
0807 18:36:35.175711 (+ 527us) negotiation.cc:107] Waiting for socket to connect
0807 18:36:35.175726 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 18:36:35.175816 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:36:35.176194 (+ 378us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:36:35.176196 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:36:35.176215 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:36:35.176358 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:36:35.176364 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:36:35.177315 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:36:35.177317 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:36:35.178209 (+ 892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:36:35.178220 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:36:35.178989 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:36:35.178992 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:36:35.179079 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:36:35.179127 (+ 48us) client_negotiation.cc:769] Sending connection context
0807 18:36:35.179195 (+ 68us) client_negotiation.cc:240] Negotiation successful
0807 18:36:35.179266 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":426,"mutex_wait_us":22,"thread_start_us":159,"threads_started":1}
I20260807 18:36:35.180155 28917 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df5fba67196f4c6892658b1580fdc44d" candidate_uuid: "f6c9cfee63a24a18b79f2e34c4222e44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8dd92e134d474cdfbb667f7435670c8d" is_pre_election: true
I20260807 18:36:35.180429 28766 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df5fba67196f4c6892658b1580fdc44d" candidate_uuid: "f6c9cfee63a24a18b79f2e34c4222e44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a890d9b2c9d44be9139631c5bdb2967" is_pre_election: true
I20260807 18:36:35.180521 28917 raft_consensus.cc:2454] T df5fba67196f4c6892658b1580fdc44d P 8dd92e134d474cdfbb667f7435670c8d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f6c9cfee63a24a18b79f2e34c4222e44 in term 0.
I20260807 18:36:35.180753 28766 raft_consensus.cc:2454] T df5fba67196f4c6892658b1580fdc44d P 8a890d9b2c9d44be9139631c5bdb2967 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f6c9cfee63a24a18b79f2e34c4222e44 in term 0.
I20260807 18:36:35.181116 28990 leader_election.cc:302] T df5fba67196f4c6892658b1580fdc44d P f6c9cfee63a24a18b79f2e34c4222e44 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8dd92e134d474cdfbb667f7435670c8d, f6c9cfee63a24a18b79f2e34c4222e44; no voters:
I20260807 18:36:35.181448 29132 raft_consensus.cc:2781] T df5fba67196f4c6892658b1580fdc44d P f6c9cfee63a24a18b79f2e34c4222e44 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:36:35.181541 29132 raft_consensus.cc:482] T df5fba67196f4c6892658b1580fdc44d P f6c9cfee63a24a18b79f2e34c4222e44 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:36:35.181586 29132 raft_consensus.cc:3037] T df5fba67196f4c6892658b1580fdc44d P f6c9cfee63a24a18b79f2e34c4222e44 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:36:35.182750 29132 raft_consensus.cc:504] T df5fba67196f4c6892658b1580fdc44d P f6c9cfee63a24a18b79f2e34c4222e44 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6c9cfee63a24a18b79f2e34c4222e44" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 39057 } } peers { permanent_uuid: "8dd92e134d474cdfbb667f7435670c8d" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 39389 } } peers { permanent_uuid: "8a890d9b2c9d44be9139631c5bdb2967" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 45483 } }
I20260807 18:36:35.183094 29132 leader_election.cc:288] T df5fba67196f4c6892658b1580fdc44d P f6c9cfee63a24a18b79f2e34c4222e44 [CANDIDATE]: Term 1 election: Requested vote from peers 8dd92e134d474cdfbb667f7435670c8d (127.11.14.66:39389), 8a890d9b2c9d44be9139631c5bdb2967 (127.11.14.65:45483)
I20260807 18:36:35.183382 28917 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df5fba67196f4c6892658b1580fdc44d" candidate_uuid: "f6c9cfee63a24a18b79f2e34c4222e44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8dd92e134d474cdfbb667f7435670c8d"
I20260807 18:36:35.183516 28917 raft_consensus.cc:3037] T df5fba67196f4c6892658b1580fdc44d P 8dd92e134d474cdfbb667f7435670c8d [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:36:35.183610 28766 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df5fba67196f4c6892658b1580fdc44d" candidate_uuid: "f6c9cfee63a24a18b79f2e34c4222e44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a890d9b2c9d44be9139631c5bdb2967"
I20260807 18:36:35.183816 28766 raft_consensus.cc:3037] T df5fba67196f4c6892658b1580fdc44d P 8a890d9b2c9d44be9139631c5bdb2967 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:36:35.184759 28917 raft_consensus.cc:2454] T df5fba67196f4c6892658b1580fdc44d P 8dd92e134d474cdfbb667f7435670c8d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f6c9cfee63a24a18b79f2e34c4222e44 in term 1.
I20260807 18:36:35.185055 28990 leader_election.cc:302] T df5fba67196f4c6892658b1580fdc44d P f6c9cfee63a24a18b79f2e34c4222e44 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8dd92e134d474cdfbb667f7435670c8d, f6c9cfee63a24a18b79f2e34c4222e44; no voters:
I20260807 18:36:35.185220 28766 raft_consensus.cc:2454] T df5fba67196f4c6892658b1580fdc44d P 8a890d9b2c9d44be9139631c5bdb2967 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f6c9cfee63a24a18b79f2e34c4222e44 in term 1.
I20260807 18:36:35.185240 29132 raft_consensus.cc:2781] T df5fba67196f4c6892658b1580fdc44d P f6c9cfee63a24a18b79f2e34c4222e44 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:36:35.185485 29132 raft_consensus.cc:686] T df5fba67196f4c6892658b1580fdc44d P f6c9cfee63a24a18b79f2e34c4222e44 [term 1 LEADER]: Becoming Leader. State: Replica: f6c9cfee63a24a18b79f2e34c4222e44, State: Running, Role: LEADER
I20260807 18:36:35.185853 29132 consensus_queue.cc:237] T df5fba67196f4c6892658b1580fdc44d P f6c9cfee63a24a18b79f2e34c4222e44 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6c9cfee63a24a18b79f2e34c4222e44" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 39057 } } peers { permanent_uuid: "8dd92e134d474cdfbb667f7435670c8d" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 39389 } } peers { permanent_uuid: "8a890d9b2c9d44be9139631c5bdb2967" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 45483 } }
I20260807 18:36:35.189458 28607 catalog_manager.cc:5495] T df5fba67196f4c6892658b1580fdc44d P f6c9cfee63a24a18b79f2e34c4222e44 reported cstate change: term changed from 0 to 1, leader changed from <none> to f6c9cfee63a24a18b79f2e34c4222e44 (127.11.14.67). New cstate: current_term: 1 leader_uuid: "f6c9cfee63a24a18b79f2e34c4222e44" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6c9cfee63a24a18b79f2e34c4222e44" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 39057 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8dd92e134d474cdfbb667f7435670c8d" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 39389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a890d9b2c9d44be9139631c5bdb2967" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 45483 } health_report { overall_health: UNKNOWN } } }
I20260807 18:36:35.230283 29124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:36:35.226511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37842 (local address 127.11.14.67:39057)
0807 18:36:35.226668 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0807 18:36:35.226673 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:36:35.226690 (+ 17us) server_negotiation.cc:407] Connection header received
0807 18:36:35.226859 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:36:35.226861 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:36:35.226920 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:36:35.227040 (+ 120us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:36:35.227317 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:36:35.227856 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:36:35.228724 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:36:35.228947 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:36:35.229062 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:36:35.229391 (+ 329us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:36:35.229833 (+ 442us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:36:35.229975 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0807 18:36:35.230037 (+ 62us) server_negotiation.cc:299] Negotiation successful
0807 18:36:35.230099 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260807 18:36:35.236651 28917 raft_consensus.cc:1261] T df5fba67196f4c6892658b1580fdc44d P 8dd92e134d474cdfbb667f7435670c8d [term 1 FOLLOWER]: Refusing update from remote peer f6c9cfee63a24a18b79f2e34c4222e44: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:36:35.236660 28766 raft_consensus.cc:1261] T df5fba67196f4c6892658b1580fdc44d P 8a890d9b2c9d44be9139631c5bdb2967 [term 1 FOLLOWER]: Refusing update from remote peer f6c9cfee63a24a18b79f2e34c4222e44: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:36:35.237452 29136 consensus_queue.cc:1035] T df5fba67196f4c6892658b1580fdc44d P f6c9cfee63a24a18b79f2e34c4222e44 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8dd92e134d474cdfbb667f7435670c8d" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 39389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:36:35.237639 29132 consensus_queue.cc:1035] T df5fba67196f4c6892658b1580fdc44d P f6c9cfee63a24a18b79f2e34c4222e44 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a890d9b2c9d44be9139631c5bdb2967" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 45483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:36:35.244836 29141 mvcc.cc:205] Tried to move back new op lower bound from 7315979449280962560 to 7315979449083985920. Current Snapshot: MvccSnapshot[applied={T|T < 7315979449280962560}]
I20260807 18:36:35.246683 29140 mvcc.cc:205] Tried to move back new op lower bound from 7315979449280962560 to 7315979449083985920. Current Snapshot: MvccSnapshot[applied={T|T < 7315979449280962560 or (T in {7315979449280962560})}]
I20260807 18:36:35.253038 28607 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50130:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:36:35.253237 28607 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50130:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:36:35.255668 28607 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9e30e67c1f6d4fb68e8f67b10bc50f96: Sending DeleteTablet for 3 replicas of tablet df5fba67196f4c6892658b1580fdc44d
I20260807 18:36:35.256362 29041 tablet_service.cc:1514] Processing DeleteTablet for tablet df5fba67196f4c6892658b1580fdc44d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:36:35 UTC) from {username='kudu', principal='kudu/127.11.14.126@KRBTEST.COM'} at 127.0.0.1:37832
I20260807 18:36:35.256489 11321 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkz56XN/build/debug/bin/kudu with pid 28651
I20260807 18:36:35.256465 28894 tablet_service.cc:1514] Processing DeleteTablet for tablet df5fba67196f4c6892658b1580fdc44d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:36:35 UTC) from {username='kudu', principal='kudu/127.11.14.126@KRBTEST.COM'} at 127.0.0.1:34904
I20260807 18:36:35.257786 29148 tablet_replica.cc:330] T df5fba67196f4c6892658b1580fdc44d P f6c9cfee63a24a18b79f2e34c4222e44: stopping tablet replica
I20260807 18:36:35.258280 29148 raft_consensus.cc:2229] T df5fba67196f4c6892658b1580fdc44d P f6c9cfee63a24a18b79f2e34c4222e44 [term 1 LEADER]: Raft consensus shutting down.
I20260807 18:36:35.258862 29148 raft_consensus.cc:2258] T df5fba67196f4c6892658b1580fdc44d P f6c9cfee63a24a18b79f2e34c4222e44 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:36:35.260741 29148 ts_tablet_manager.cc:1905] T df5fba67196f4c6892658b1580fdc44d P f6c9cfee63a24a18b79f2e34c4222e44: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:36:35.263800 29149 tablet_replica.cc:330] T df5fba67196f4c6892658b1580fdc44d P 8dd92e134d474cdfbb667f7435670c8d: stopping tablet replica
I20260807 18:36:35.264189 29149 raft_consensus.cc:2229] T df5fba67196f4c6892658b1580fdc44d P 8dd92e134d474cdfbb667f7435670c8d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:36:35.264532 29148 ts_tablet_manager.cc:1918] T df5fba67196f4c6892658b1580fdc44d P f6c9cfee63a24a18b79f2e34c4222e44: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:36:35.264647 29148 log.cc:1196] T df5fba67196f4c6892658b1580fdc44d P f6c9cfee63a24a18b79f2e34c4222e44: Deleting WAL directory at /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/minicluster-data/ts-2/wal/wals/df5fba67196f4c6892658b1580fdc44d
I20260807 18:36:35.264668 29149 raft_consensus.cc:2258] T df5fba67196f4c6892658b1580fdc44d P 8dd92e134d474cdfbb667f7435670c8d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:36:35.265023 29148 ts_tablet_manager.cc:1939] T df5fba67196f4c6892658b1580fdc44d P f6c9cfee63a24a18b79f2e34c4222e44: Deleting consensus metadata
I20260807 18:36:35.266304 29149 ts_tablet_manager.cc:1905] T df5fba67196f4c6892658b1580fdc44d P 8dd92e134d474cdfbb667f7435670c8d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:36:35.266774 28588 catalog_manager.cc:4841] TS f6c9cfee63a24a18b79f2e34c4222e44 (127.11.14.67:39057): tablet df5fba67196f4c6892658b1580fdc44d (table test-table [id=fc006f47ca0741f48209f016cd610f7d]) successfully deleted
I20260807 18:36:35.268996 29149 ts_tablet_manager.cc:1918] T df5fba67196f4c6892658b1580fdc44d P 8dd92e134d474cdfbb667f7435670c8d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:36:35.269076 29149 log.cc:1196] T df5fba67196f4c6892658b1580fdc44d P 8dd92e134d474cdfbb667f7435670c8d: Deleting WAL directory at /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/minicluster-data/ts-1/wal/wals/df5fba67196f4c6892658b1580fdc44d
I20260807 18:36:35.269356 29149 ts_tablet_manager.cc:1939] T df5fba67196f4c6892658b1580fdc44d P 8dd92e134d474cdfbb667f7435670c8d: Deleting consensus metadata
W20260807 18:36:35.269778 28590 connection.cc:535] client connection to 127.11.14.65:45483 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260807 18:36:35.269940 28590 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260807 18:36:35.270198 11321 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkz56XN/build/debug/bin/kudu with pid 28819
I20260807 18:36:35.280728 28823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:36:35.280299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.65:45483 (local address 127.0.0.1:41540)
0807 18:36:35.280488 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0807 18:36:35.280592 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.65:45483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
W20260807 18:36:35.280978 28590 catalog_manager.cc:4584] TS 8a890d9b2c9d44be9139631c5bdb2967 (127.11.14.65:45483): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet df5fba67196f4c6892658b1580fdc44d: Network error: Client connection negotiation failed: client connection to 127.11.14.65:45483: connect: Connection refused (error 111)
I20260807 18:36:35.283356 11321 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkz56XN/build/debug/bin/kudu with pid 28971
I20260807 18:36:35.287225 28823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:36:35.283536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.66:39389 (local address 127.0.0.1:34916)
0807 18:36:35.287081 (+ 3545us) negotiation.cc:107] Waiting for socket to connect
0807 18:36:35.287145 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.66:39389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":3428}
I20260807 18:36:35.292716 11321 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkz56XN/build/debug/bin/kudu with pid 28569
I20260807 18:36:55.301407 11321 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkz56XN/build/debug/bin/kudu
/tmp/dist-test-taskkz56XN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.14.126:40109
--webserver_interface=127.11.14.126
--webserver_port=41133
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.126.keytab
--principal=kudu/127.11.14.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.14.84:35419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.14.126:40109
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:36:55.419485 29152 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:36:55.419761 29152 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:36:55.419854 29152 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:36:55.423785 29152 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260807 18:36:55.423840 29152 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260807 18:36:55.423856 29152 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260807 18:36:55.423870 29152 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:36:55.423929 29152 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:36:55.423964 29152 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:36:55.423991 29152 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:36:55.424018 29152 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:36:55.429251 29152 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.14.84:35419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.11.14.126:40109
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.126.keytab
--principal=kudu/127.11.14.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.14.126:40109
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.14.126
--webserver_port=41133
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29152
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:36:55.430292 29152 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:36:55.431301 29152 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:36:55.440026 29160 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:36:55.439996 29158 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:36:55.440027 29157 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:36:55.440271 29152 server_base.cc:1034] running on GCE node
I20260807 18:36:55.440936 29152 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:36:55.442094 29152 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:36:55.443255 29152 hybrid_clock.cc:648] HybridClock initialized: now 1786127815443243 us; error 28 us; skew 500 ppm
Aug 07 18:36:55 dist-test-slave-6k22 krb5kdc[27955](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127815, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:36:55.452906 29152 init.cc:377] Logged in from keytab as kudu/127.11.14.126@KRBTEST.COM (short username kudu)
I20260807 18:36:55.453881 29152 webserver.cc:460] Webserver started at http://127.11.14.126:41133/ using document root <none> and password file <none>
I20260807 18:36:55.454478 29152 fs_manager.cc:359] Metadata directory not provided
I20260807 18:36:55.454540 29152 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:36:55.458474 29152 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260807 18:36:55.460796 29167 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:36:55.462095 29152 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 18:36:55.462229 29152 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/minicluster-data/master-0/data,/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/minicluster-data/master-0/wal
uuid: "9e30e67c1f6d4fb68e8f67b10bc50f96"
format_stamp: "Formatted at 2026-08-07 18:36:13 on dist-test-slave-6k22"
server_key: "a80393781a76b348478afae6879517bd"
server_key_iv: "de5d6f34fc2b7da1a136cd32c644d014"
server_key_version: "encryptionkey@0"
I20260807 18:36:55.462751 29152 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:36:55.473137 29152 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:36:55.476312 29152 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:36:55.476516 29152 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:36:55.484113 29152 rpc_server.cc:307] RPC server started. Bound to: 127.11.14.126:40109
I20260807 18:36:55.484426 29222 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.14.126:40109 every 8 connection(s)
I20260807 18:36:55.485226 29152 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/minicluster-data/master-0/data/info.pb
I20260807 18:36:55.489806 29223 sys_catalog.cc:263] Verifying existing consensus state
I20260807 18:36:55.490094 11321 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkz56XN/build/debug/bin/kudu as pid 29152
I20260807 18:36:55.490636 11321 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkz56XN/build/debug/bin/kudu
/tmp/dist-test-taskkz56XN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.14.65:45483
--local_ip_for_outbound_sockets=127.11.14.65
--tserver_master_addrs=127.11.14.126:40109
--webserver_port=34441
--webserver_interface=127.11.14.65
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.65.keytab
--principal=kudu/127.11.14.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.14.84:35419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 18:36:55.492254 29223 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9e30e67c1f6d4fb68e8f67b10bc50f96: Bootstrap starting.
I20260807 18:36:55.510572 29223 log.cc:824] T 00000000000000000000000000000000 P 9e30e67c1f6d4fb68e8f67b10bc50f96: Log is configured to *not* fsync() on all Append() calls
I20260807 18:36:55.523197 29223 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9e30e67c1f6d4fb68e8f67b10bc50f96: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 18:36:55.523715 29223 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9e30e67c1f6d4fb68e8f67b10bc50f96: Bootstrap complete.
I20260807 18:36:55.528044 29223 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9e30e67c1f6d4fb68e8f67b10bc50f96 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e30e67c1f6d4fb68e8f67b10bc50f96" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 40109 } }
I20260807 18:36:55.528450 29223 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9e30e67c1f6d4fb68e8f67b10bc50f96 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e30e67c1f6d4fb68e8f67b10bc50f96, State: Initialized, Role: FOLLOWER
I20260807 18:36:55.528964 29223 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9e30e67c1f6d4fb68e8f67b10bc50f96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e30e67c1f6d4fb68e8f67b10bc50f96" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 40109 } }
I20260807 18:36:55.529116 29223 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9e30e67c1f6d4fb68e8f67b10bc50f96 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:36:55.529218 29223 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9e30e67c1f6d4fb68e8f67b10bc50f96 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:36:55.529340 29223 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9e30e67c1f6d4fb68e8f67b10bc50f96 [term 2 FOLLOWER]: Advancing to term 3
I20260807 18:36:55.531309 29223 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9e30e67c1f6d4fb68e8f67b10bc50f96 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e30e67c1f6d4fb68e8f67b10bc50f96" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 40109 } }
I20260807 18:36:55.531719 29223 leader_election.cc:302] T 00000000000000000000000000000000 P 9e30e67c1f6d4fb68e8f67b10bc50f96 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9e30e67c1f6d4fb68e8f67b10bc50f96; no voters:
I20260807 18:36:55.532079 29223 leader_election.cc:288] T 00000000000000000000000000000000 P 9e30e67c1f6d4fb68e8f67b10bc50f96 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260807 18:36:55.532174 29227 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9e30e67c1f6d4fb68e8f67b10bc50f96 [term 3 FOLLOWER]: Leader election won for term 3
I20260807 18:36:55.532383 29227 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9e30e67c1f6d4fb68e8f67b10bc50f96 [term 3 LEADER]: Becoming Leader. State: Replica: 9e30e67c1f6d4fb68e8f67b10bc50f96, State: Running, Role: LEADER
I20260807 18:36:55.532724 29227 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9e30e67c1f6d4fb68e8f67b10bc50f96 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e30e67c1f6d4fb68e8f67b10bc50f96" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 40109 } }
I20260807 18:36:55.533423 29223 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9e30e67c1f6d4fb68e8f67b10bc50f96 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:36:55.533747 29227 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9e30e67c1f6d4fb68e8f67b10bc50f96 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9e30e67c1f6d4fb68e8f67b10bc50f96. Latest consensus state: current_term: 3 leader_uuid: "9e30e67c1f6d4fb68e8f67b10bc50f96" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e30e67c1f6d4fb68e8f67b10bc50f96" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 40109 } } }
I20260807 18:36:55.533877 29227 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9e30e67c1f6d4fb68e8f67b10bc50f96 [sys.catalog]: This master's current role is: LEADER
I20260807 18:36:55.534253 29228 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9e30e67c1f6d4fb68e8f67b10bc50f96 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "9e30e67c1f6d4fb68e8f67b10bc50f96" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e30e67c1f6d4fb68e8f67b10bc50f96" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 40109 } } }
I20260807 18:36:55.534497 29228 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9e30e67c1f6d4fb68e8f67b10bc50f96 [sys.catalog]: This master's current role is: LEADER
I20260807 18:36:55.534255 29232 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:36:55.540129 29232 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:36:55.540899 29232 catalog_manager.cc:1259] Loaded cluster ID: 6f64b49830594df991154b1ee8acec8f
I20260807 18:36:55.540962 29232 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:36:55.543428 29232 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:36:55.544174 29232 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 9e30e67c1f6d4fb68e8f67b10bc50f96: Loaded TSK: 0
I20260807 18:36:55.544257 29232 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 9e30e67c1f6d4fb68e8f67b10bc50f96: Loaded TSK: 1
F20260807 18:36:55.544982 29232 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f2c82a369ad google::LogMessage::Fail() at ??:0
@ 0x7f2c82a3988a google::LogMessage::SendToLog() at ??:0
@ 0x7f2c82a364de google::LogMessage::Flush() at ??:0
@ 0x7f2c82a37a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f2c856fa18d kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f2c856f2f0d kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f2c8d39b7bb kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f2c8d37599c kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f2c8d376178 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlvE3_clEv at ??:0
@ 0x7f2c8d3b7df8 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7f2c8d3d682b std::function<>::operator()() at ??:0
@ 0x7f2c8d375b4f _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7f2c8d376f3e kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f2c8d373579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7f2c8d3b7640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f2c84c883d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f2c84c88c83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7f2c84c8a3f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f2c84c79512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f2c862bd6db start_thread at ??:0
@ 0x7f2c8153461f clone at ??:0
W20260807 18:36:55.651922 29225 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:36:55.652165 29225 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:36:55.652209 29225 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:36:55.656315 29225 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:36:55.656371 29225 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:36:55.656459 29225 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.14.65
I20260807 18:36:55.661805 29225 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.14.84:35419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.65.keytab
--principal=kudu/127.11.14.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.14.65:45483
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.14.65
--webserver_port=34441
--tserver_master_addrs=127.11.14.126:40109
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29225
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.14.65
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:36:55.662863 29225 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:36:55.663942 29225 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:36:55.674258 29277 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:36:55.674253 29276 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:36:55.674224 29279 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:36:55.674643 29225 server_base.cc:1034] running on GCE node
I20260807 18:36:55.675081 29225 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:36:55.675863 29225 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:36:55.677066 29225 hybrid_clock.cc:648] HybridClock initialized: now 1786127815677038 us; error 42 us; skew 500 ppm
Aug 07 18:36:55 dist-test-slave-6k22 krb5kdc[27955](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127815, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:36:55.680055 29225 init.cc:377] Logged in from keytab as kudu/127.11.14.65@KRBTEST.COM (short username kudu)
I20260807 18:36:55.680960 29225 webserver.cc:460] Webserver started at http://127.11.14.65:34441/ using document root <none> and password file <none>
I20260807 18:36:55.681556 29225 fs_manager.cc:359] Metadata directory not provided
I20260807 18:36:55.681620 29225 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:36:55.685830 29225 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 18:36:55.688235 29287 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:36:55.689610 29225 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:36:55.689751 29225 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/minicluster-data/ts-0/data,/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/minicluster-data/ts-0/wal
uuid: "8a890d9b2c9d44be9139631c5bdb2967"
format_stamp: "Formatted at 2026-08-07 18:36:13 on dist-test-slave-6k22"
server_key: "1ff1f02b7e55fca2ea09aa675715ea75"
server_key_iv: "c3fe88fa8a4dabab522d6eec85ec86e9"
server_key_version: "encryptionkey@0"
I20260807 18:36:55.690311 29225 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:36:55.708874 29225 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:36:55.712415 29225 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:36:55.712632 29225 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:36:55.713263 29225 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:36:55.714437 29294 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260807 18:36:55.720194 29225 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260807 18:36:55.720286 29225 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.000s sys 0.000s
I20260807 18:36:55.720374 29225 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260807 18:36:55.722190 29225 ts_tablet_manager.cc:610] Registered 1 tablets
I20260807 18:36:55.722265 29225 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260807 18:36:55.722577 29294 tablet_bootstrap.cc:492] T df5fba67196f4c6892658b1580fdc44d P 8a890d9b2c9d44be9139631c5bdb2967: Bootstrap starting.
I20260807 18:36:55.733609 29225 rpc_server.cc:307] RPC server started. Bound to: 127.11.14.65:45483
I20260807 18:36:55.733837 29415 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.14.65:45483 every 8 connection(s)
I20260807 18:36:55.734658 29225 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/minicluster-data/ts-0/data/info.pb
I20260807 18:36:55.739161 11321 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkz56XN/build/debug/bin/kudu as pid 29225
I20260807 18:36:55.739698 11321 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkz56XN/build/debug/bin/kudu
/tmp/dist-test-taskkz56XN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.14.66:39389
--local_ip_for_outbound_sockets=127.11.14.66
--tserver_master_addrs=127.11.14.126:40109
--webserver_port=36717
--webserver_interface=127.11.14.66
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.66.keytab
--principal=kudu/127.11.14.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.14.84:35419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 18:36:55.747428 29294 log.cc:824] T df5fba67196f4c6892658b1580fdc44d P 8a890d9b2c9d44be9139631c5bdb2967: Log is configured to *not* fsync() on all Append() calls
I20260807 18:36:55.754670 29294 tablet_bootstrap.cc:492] T df5fba67196f4c6892658b1580fdc44d P 8a890d9b2c9d44be9139631c5bdb2967: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 18:36:55.755074 29294 tablet_bootstrap.cc:492] T df5fba67196f4c6892658b1580fdc44d P 8a890d9b2c9d44be9139631c5bdb2967: Bootstrap complete.
I20260807 18:36:55.756408 29294 ts_tablet_manager.cc:1397] T df5fba67196f4c6892658b1580fdc44d P 8a890d9b2c9d44be9139631c5bdb2967: Time spent bootstrapping tablet: real 0.034s user 0.021s sys 0.010s
I20260807 18:36:55.758833 29294 raft_consensus.cc:348] T df5fba67196f4c6892658b1580fdc44d P 8a890d9b2c9d44be9139631c5bdb2967 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6c9cfee63a24a18b79f2e34c4222e44" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 39057 } } peers { permanent_uuid: "8dd92e134d474cdfbb667f7435670c8d" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 39389 } } peers { permanent_uuid: "8a890d9b2c9d44be9139631c5bdb2967" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 45483 } }
I20260807 18:36:55.759277 29294 raft_consensus.cc:729] T df5fba67196f4c6892658b1580fdc44d P 8a890d9b2c9d44be9139631c5bdb2967 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a890d9b2c9d44be9139631c5bdb2967, State: Initialized, Role: FOLLOWER
I20260807 18:36:55.759815 29294 consensus_queue.cc:260] T df5fba67196f4c6892658b1580fdc44d P 8a890d9b2c9d44be9139631c5bdb2967 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6c9cfee63a24a18b79f2e34c4222e44" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 39057 } } peers { permanent_uuid: "8dd92e134d474cdfbb667f7435670c8d" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 39389 } } peers { permanent_uuid: "8a890d9b2c9d44be9139631c5bdb2967" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 45483 } }
I20260807 18:36:55.760864 29294 ts_tablet_manager.cc:1428] T df5fba67196f4c6892658b1580fdc44d P 8a890d9b2c9d44be9139631c5bdb2967: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
W20260807 18:36:55.784643 29418 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:36:55.736713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:40109 (local address 127.11.14.65:38093)
0807 18:36:55.737245 (+ 532us) negotiation.cc:107] Waiting for socket to connect
0807 18:36:55.737300 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0807 18:36:55.738177 (+ 877us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:36:55.783856 (+ 45679us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.126:40109: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":302,"thread_start_us":168,"threads_started":1}
W20260807 18:36:55.785313 29416 heartbeater.cc:644] Failed to heartbeat to 127.11.14.126:40109 (0 consecutive failures): Network error: Failed to ping master at 127.11.14.126:40109: Client connection negotiation failed: client connection to 127.11.14.126:40109: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260807 18:36:55.786125 29418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:36:55.785853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:40109 (local address 127.11.14.65:42987)
0807 18:36:55.785991 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0807 18:36:55.786061 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.126:40109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260807 18:36:55.787797 29418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:36:55.787455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:40109 (local address 127.11.14.65:52003)
0807 18:36:55.787614 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0807 18:36:55.787706 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.126:40109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260807 18:36:55.788136 29416 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 18:36:55.887253 29419 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:36:55.887542 29419 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:36:55.887633 29419 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:36:55.891741 29419 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:36:55.891813 29419 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:36:55.891901 29419 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.14.66
I20260807 18:36:55.897271 29419 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.14.84:35419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.66.keytab
--principal=kudu/127.11.14.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.14.66:39389
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.14.66
--webserver_port=36717
--tserver_master_addrs=127.11.14.126:40109
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29419
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.14.66
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:36:55.898331 29419 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:36:55.899433 29419 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:36:55.908999 29426 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:36:55.909082 29430 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:36:55.909070 29427 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:36:55.909370 29419 server_base.cc:1034] running on GCE node
I20260807 18:36:55.909762 29419 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:36:55.910379 29419 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:36:55.911612 29419 hybrid_clock.cc:648] HybridClock initialized: now 1786127815911586 us; error 32 us; skew 500 ppm
Aug 07 18:36:55 dist-test-slave-6k22 krb5kdc[27955](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127815, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:36:55.914346 29419 init.cc:377] Logged in from keytab as kudu/127.11.14.66@KRBTEST.COM (short username kudu)
I20260807 18:36:55.915230 29419 webserver.cc:460] Webserver started at http://127.11.14.66:36717/ using document root <none> and password file <none>
I20260807 18:36:55.915889 29419 fs_manager.cc:359] Metadata directory not provided
I20260807 18:36:55.915984 29419 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:36:55.919806 29419 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.004s sys 0.000s
I20260807 18:36:55.921859 29437 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:36:55.923079 29419 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 18:36:55.923307 29419 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/minicluster-data/ts-1/data,/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/minicluster-data/ts-1/wal
uuid: "8dd92e134d474cdfbb667f7435670c8d"
format_stamp: "Formatted at 2026-08-07 18:36:13 on dist-test-slave-6k22"
server_key: "f57f065283ab1a4e5f3b246de1834c43"
server_key_iv: "0653dbc1ced63c01d8f6843d603fe578"
server_key_version: "encryptionkey@0"
I20260807 18:36:55.924023 29419 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:36:55.941805 29419 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:36:55.945407 29419 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:36:55.945647 29419 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:36:55.946312 29419 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:36:55.947191 29419 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:36:55.947264 29419 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:36:55.947352 29419 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:36:55.947408 29419 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:36:55.957024 29419 rpc_server.cc:307] RPC server started. Bound to: 127.11.14.66:39389
I20260807 18:36:55.957072 29563 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.14.66:39389 every 8 connection(s)
I20260807 18:36:55.958101 29419 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/minicluster-data/ts-1/data/info.pb
I20260807 18:36:55.960479 11321 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkz56XN/build/debug/bin/kudu as pid 29419
I20260807 18:36:55.960985 11321 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkz56XN/build/debug/bin/kudu
/tmp/dist-test-taskkz56XN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.14.67:39057
--local_ip_for_outbound_sockets=127.11.14.67
--tserver_master_addrs=127.11.14.126:40109
--webserver_port=38235
--webserver_interface=127.11.14.67
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.67.keytab
--principal=kudu/127.11.14.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.14.84:35419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 18:36:55.962023 29566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:36:55.960390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:40109 (local address 127.11.14.66:59029)
0807 18:36:55.961163 (+ 773us) negotiation.cc:107] Waiting for socket to connect
0807 18:36:55.961293 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.126:40109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":558,"thread_start_us":144,"threads_started":1}
W20260807 18:36:55.962495 29564 heartbeater.cc:644] Failed to heartbeat to 127.11.14.126:40109 (0 consecutive failures): Network error: Failed to ping master at 127.11.14.126:40109: Client connection negotiation failed: client connection to 127.11.14.126:40109: connect: Connection refused (error 111)
I20260807 18:36:55.963243 29566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:36:55.962955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:40109 (local address 127.11.14.66:56437)
0807 18:36:55.963097 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0807 18:36:55.963161 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.126:40109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260807 18:36:55.963999 29566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:36:55.963749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:40109 (local address 127.11.14.66:58121)
0807 18:36:55.963879 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0807 18:36:55.963929 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.126:40109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260807 18:36:55.964188 29564 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 18:36:56.087517 29567 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:36:56.087815 29567 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:36:56.087901 29567 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:36:56.092216 29567 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:36:56.092271 29567 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:36:56.092367 29567 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.14.67
I20260807 18:36:56.097787 29567 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.14.84:35419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.67.keytab
--principal=kudu/127.11.14.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.14.67:39057
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.14.67
--webserver_port=38235
--tserver_master_addrs=127.11.14.126:40109
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29567
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.14.67
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:36:56.098878 29567 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:36:56.099963 29567 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:36:56.109892 29572 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:36:56.109968 29576 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:36:56.109972 29574 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:36:56.110759 29567 server_base.cc:1034] running on GCE node
I20260807 18:36:56.111186 29567 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:36:56.111948 29567 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:36:56.113145 29567 hybrid_clock.cc:648] HybridClock initialized: now 1786127816113127 us; error 28 us; skew 500 ppm
Aug 07 18:36:56 dist-test-slave-6k22 krb5kdc[27955](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127816, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:36:56.115890 29567 init.cc:377] Logged in from keytab as kudu/127.11.14.67@KRBTEST.COM (short username kudu)
I20260807 18:36:56.116824 29567 webserver.cc:460] Webserver started at http://127.11.14.67:38235/ using document root <none> and password file <none>
I20260807 18:36:56.117424 29567 fs_manager.cc:359] Metadata directory not provided
I20260807 18:36:56.117525 29567 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:36:56.121546 29567 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 18:36:56.123744 29584 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:36:56.124965 29567 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 18:36:56.125108 29567 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/minicluster-data/ts-2/data,/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/minicluster-data/ts-2/wal
uuid: "f6c9cfee63a24a18b79f2e34c4222e44"
format_stamp: "Formatted at 2026-08-07 18:36:13 on dist-test-slave-6k22"
server_key: "1f168783971fc54e3f339eabb2b63d2b"
server_key_iv: "9d915462a34803bbef932b44281d251e"
server_key_version: "encryptionkey@0"
I20260807 18:36:56.125676 29567 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:36:56.147338 29567 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:36:56.150454 29567 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:36:56.150666 29567 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:36:56.151285 29567 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:36:56.152230 29567 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:36:56.152312 29567 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:36:56.152392 29567 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:36:56.152429 29567 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:36:56.161684 29567 rpc_server.cc:307] RPC server started. Bound to: 127.11.14.67:39057
I20260807 18:36:56.161710 29708 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.14.67:39057 every 8 connection(s)
I20260807 18:36:56.162698 29567 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786127579900523-11321-0/minicluster-data/ts-2/data/info.pb
I20260807 18:36:56.166436 29711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:36:56.164930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:40109 (local address 127.11.14.67:33727)
0807 18:36:56.165599 (+ 669us) negotiation.cc:107] Waiting for socket to connect
0807 18:36:56.165703 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.126:40109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":476,"thread_start_us":110,"threads_started":1}
W20260807 18:36:56.166883 29709 heartbeater.cc:644] Failed to heartbeat to 127.11.14.126:40109 (0 consecutive failures): Network error: Failed to ping master at 127.11.14.126:40109: Client connection negotiation failed: client connection to 127.11.14.126:40109: connect: Connection refused (error 111)
I20260807 18:36:56.167485 29711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:36:56.167255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:40109 (local address 127.11.14.67:59289)
0807 18:36:56.167377 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0807 18:36:56.167433 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.126:40109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260807 18:36:56.168107 29711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:36:56.167901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:40109 (local address 127.11.14.67:33601)
0807 18:36:56.168004 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0807 18:36:56.168049 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.126:40109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260807 18:36:56.168285 29709 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260807 18:36:56.169425 11321 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkz56XN/build/debug/bin/kudu as pid 29567
I20260807 18:36:56.789245 29713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:36:56.788805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:40109 (local address 127.11.14.65:43427)
0807 18:36:56.789068 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0807 18:36:56.789127 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.126:40109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":81,"threads_started":1}
I20260807 18:36:56.965196 29714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:36:56.964745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:40109 (local address 127.11.14.66:50441)
0807 18:36:56.965027 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0807 18:36:56.965086 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.126:40109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":90,"threads_started":1}
I20260807 18:36:57.144742 29715 raft_consensus.cc:482] T df5fba67196f4c6892658b1580fdc44d P 8a890d9b2c9d44be9139631c5bdb2967 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:36:57.144992 29715 raft_consensus.cc:504] T df5fba67196f4c6892658b1580fdc44d P 8a890d9b2c9d44be9139631c5bdb2967 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6c9cfee63a24a18b79f2e34c4222e44" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 39057 } } peers { permanent_uuid: "8dd92e134d474cdfbb667f7435670c8d" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 39389 } } peers { permanent_uuid: "8a890d9b2c9d44be9139631c5bdb2967" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 45483 } }
I20260807 18:36:57.146149 29715 leader_election.cc:288] T df5fba67196f4c6892658b1580fdc44d P 8a890d9b2c9d44be9139631c5bdb2967 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f6c9cfee63a24a18b79f2e34c4222e44 (127.11.14.67:39057), 8dd92e134d474cdfbb667f7435670c8d (127.11.14.66:39389)
Aug 07 18:36:57 dist-test-slave-6k22 krb5kdc[27955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127815, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.65@KRBTEST.COM for kudu/127.11.14.67@KRBTEST.COM
Aug 07 18:36:57 dist-test-slave-6k22 krb5kdc[27955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127815, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.65@KRBTEST.COM for kudu/127.11.14.66@KRBTEST.COM
I20260807 18:36:57.161170 29718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:36:57.146984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.65:58599 (local address 127.11.14.66:39389)
0807 18:36:57.147328 (+ 344us) server_negotiation.cc:206] Beginning negotiation
0807 18:36:57.147349 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0807 18:36:57.147383 (+ 34us) server_negotiation.cc:407] Connection header received
0807 18:36:57.148093 (+ 710us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:36:57.148099 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:36:57.148409 (+ 310us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:36:57.148719 (+ 310us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:36:57.150054 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:36:57.151035 (+ 981us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:36:57.152469 (+ 1434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:36:57.152927 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:36:57.153074 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:36:57.156530 (+ 3456us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:36:57.156535 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:36:57.156545 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:36:57.156572 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:36:57.158337 (+ 1765us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:36:57.159031 (+ 694us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:36:57.159036 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:36:57.159040 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:36:57.159144 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:36:57.159628 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:36:57.159631 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:36:57.159633 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:36:57.160065 (+ 432us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:36:57.160193 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0807 18:36:57.160728 (+ 535us) server_negotiation.cc:299] Negotiation successful
0807 18:36:57.160973 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":238,"thread_start_us":100,"threads_started":1}
I20260807 18:36:57.161183 29713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:36:57.146386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.67:39057 (local address 127.11.14.65:52923)
0807 18:36:57.146558 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0807 18:36:57.146578 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 18:36:57.146728 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:36:57.148779 (+ 2051us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:36:57.148785 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:36:57.149382 (+ 597us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:36:57.149870 (+ 488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:36:57.149884 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:36:57.151326 (+ 1442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:36:57.151329 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:36:57.152335 (+ 1006us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:36:57.152345 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:36:57.153118 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:36:57.153121 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:36:57.153309 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:36:57.154059 (+ 750us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:36:57.154082 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:36:57.155886 (+ 1804us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:36:57.158523 (+ 2637us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:36:57.158525 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:36:57.158527 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:36:57.158879 (+ 352us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:36:57.159280 (+ 401us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:36:57.159282 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:36:57.159283 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:36:57.159419 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:36:57.160229 (+ 810us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:36:57.160232 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:36:57.160422 (+ 190us) client_negotiation.cc:769] Sending connection context
0807 18:36:57.160617 (+ 195us) client_negotiation.cc:240] Negotiation successful
0807 18:36:57.160963 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":60,"mutex_wait_us":169,"spinlock_wait_cycles":9728}
I20260807 18:36:57.161433 29716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:36:57.146603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.65:52923 (local address 127.11.14.67:39057)
0807 18:36:57.147031 (+ 428us) server_negotiation.cc:206] Beginning negotiation
0807 18:36:57.147051 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0807 18:36:57.147107 (+ 56us) server_negotiation.cc:407] Connection header received
0807 18:36:57.147975 (+ 868us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:36:57.147982 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:36:57.148396 (+ 414us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:36:57.148671 (+ 275us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:36:57.150084 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:36:57.151151 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:36:57.152470 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:36:57.152964 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:36:57.153161 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:36:57.156134 (+ 2973us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:36:57.156142 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:36:57.156159 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:36:57.156196 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:36:57.158387 (+ 2191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:36:57.158998 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:36:57.159007 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:36:57.159014 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:36:57.159149 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:36:57.159528 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:36:57.159535 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:36:57.159536 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:36:57.160085 (+ 549us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:36:57.160245 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0807 18:36:57.160752 (+ 507us) server_negotiation.cc:299] Negotiation successful
0807 18:36:57.161054 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":292,"thread_start_us":126,"threads_started":1}
I20260807 18:36:57.162217 29717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:36:57.146698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.66:39389 (local address 127.11.14.65:58599)
0807 18:36:57.147013 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0807 18:36:57.147024 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 18:36:57.147103 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:36:57.149072 (+ 1969us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:36:57.149075 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:36:57.149379 (+ 304us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:36:57.149870 (+ 491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:36:57.149885 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:36:57.151196 (+ 1311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:36:57.151201 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:36:57.152333 (+ 1132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:36:57.152344 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:36:57.153058 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:36:57.153065 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:36:57.153295 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:36:57.154127 (+ 832us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:36:57.154153 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:36:57.156351 (+ 2198us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:36:57.158475 (+ 2124us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:36:57.158485 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:36:57.158502 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:36:57.158888 (+ 386us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:36:57.159249 (+ 361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:36:57.159254 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:36:57.159257 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:36:57.159429 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:36:57.160195 (+ 766us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:36:57.160202 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:36:57.160425 (+ 223us) client_negotiation.cc:769] Sending connection context
0807 18:36:57.160619 (+ 194us) client_negotiation.cc:240] Negotiation successful
0807 18:36:57.161058 (+ 439us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":253,"mutex_wait_us":300,"spinlock_wait_cycles":53376,"thread_start_us":77,"threads_started":1}
W20260807 18:36:57.164175 29288 leader_election.cc:341] T df5fba67196f4c6892658b1580fdc44d P 8a890d9b2c9d44be9139631c5bdb2967 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f6c9cfee63a24a18b79f2e34c4222e44 (127.11.14.67:39057): Not found: Tablet not found: df5fba67196f4c6892658b1580fdc44d
W20260807 18:36:57.165655 29288 leader_election.cc:341] T df5fba67196f4c6892658b1580fdc44d P 8a890d9b2c9d44be9139631c5bdb2967 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8dd92e134d474cdfbb667f7435670c8d (127.11.14.66:39389): Not found: Tablet not found: df5fba67196f4c6892658b1580fdc44d
I20260807 18:36:57.165761 29288 leader_election.cc:302] T df5fba67196f4c6892658b1580fdc44d P 8a890d9b2c9d44be9139631c5bdb2967 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a890d9b2c9d44be9139631c5bdb2967; no voters: 8dd92e134d474cdfbb667f7435670c8d, f6c9cfee63a24a18b79f2e34c4222e44
I20260807 18:36:57.166004 29715 raft_consensus.cc:2726] T df5fba67196f4c6892658b1580fdc44d P 8a890d9b2c9d44be9139631c5bdb2967 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 18:36:57.169090 29719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:36:57.168751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:40109 (local address 127.11.14.67:41787)
0807 18:36:57.168959 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0807 18:36:57.169012 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.126:40109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":62,"threads_started":1}
I20260807 18:36:57.790501 29720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:36:57.790076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:40109 (local address 127.11.14.65:58985)
0807 18:36:57.790347 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0807 18:36:57.790403 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.126:40109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":80,"threads_started":1}
I20260807 18:36:57.966517 29721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:36:57.966066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:40109 (local address 127.11.14.66:54617)
0807 18:36:57.966370 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0807 18:36:57.966427 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.126:40109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":81,"threads_started":1}
I20260807 18:36:58.170189 29722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:36:58.169778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:40109 (local address 127.11.14.67:33763)
0807 18:36:58.170021 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0807 18:36:58.170079 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.126:40109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":69,"threads_started":1}
I20260807 18:36:58.791803 29723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:36:58.791322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:40109 (local address 127.11.14.65:55569)
0807 18:36:58.791639 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0807 18:36:58.791696 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.126:40109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":79,"threads_started":1}
I20260807 18:36:58.967965 29724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:36:58.967368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:40109 (local address 127.11.14.66:40081)
0807 18:36:58.967780 (+ 412us) negotiation.cc:107] Waiting for socket to connect
0807 18:36:58.967846 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.126:40109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":321,"thread_start_us":72,"threads_started":1}
I20260807 18:36:59.171384 29725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:36:59.170962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:40109 (local address 127.11.14.67:49053)
0807 18:36:59.171234 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0807 18:36:59.171295 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.126:40109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":91,"threads_started":1}
I20260807 18:36:59.235255 29726 raft_consensus.cc:482] T df5fba67196f4c6892658b1580fdc44d P 8a890d9b2c9d44be9139631c5bdb2967 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:36:59.235400 29726 raft_consensus.cc:504] T df5fba67196f4c6892658b1580fdc44d P 8a890d9b2c9d44be9139631c5bdb2967 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6c9cfee63a24a18b79f2e34c4222e44" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 39057 } } peers { permanent_uuid: "8dd92e134d474cdfbb667f7435670c8d" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 39389 } } peers { permanent_uuid: "8a890d9b2c9d44be9139631c5bdb2967" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 45483 } }
I20260807 18:36:59.235873 29726 leader_election.cc:288] T df5fba67196f4c6892658b1580fdc44d P 8a890d9b2c9d44be9139631c5bdb2967 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f6c9cfee63a24a18b79f2e34c4222e44 (127.11.14.67:39057), 8dd92e134d474cdfbb667f7435670c8d (127.11.14.66:39389)
W20260807 18:36:59.236747 29288 leader_election.cc:341] T df5fba67196f4c6892658b1580fdc44d P 8a890d9b2c9d44be9139631c5bdb2967 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f6c9cfee63a24a18b79f2e34c4222e44 (127.11.14.67:39057): Not found: Tablet not found: df5fba67196f4c6892658b1580fdc44d
W20260807 18:36:59.236934 29288 leader_election.cc:341] T df5fba67196f4c6892658b1580fdc44d P 8a890d9b2c9d44be9139631c5bdb2967 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8dd92e134d474cdfbb667f7435670c8d (127.11.14.66:39389): Not found: Tablet not found: df5fba67196f4c6892658b1580fdc44d
I20260807 18:36:59.237001 29288 leader_election.cc:302] T df5fba67196f4c6892658b1580fdc44d P 8a890d9b2c9d44be9139631c5bdb2967 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a890d9b2c9d44be9139631c5bdb2967; no voters: 8dd92e134d474cdfbb667f7435670c8d, f6c9cfee63a24a18b79f2e34c4222e44
I20260807 18:36:59.237188 29726 raft_consensus.cc:2726] T df5fba67196f4c6892658b1580fdc44d P 8a890d9b2c9d44be9139631c5bdb2967 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 18:36:59.793002 29727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:36:59.792573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:40109 (local address 127.11.14.65:46563)
0807 18:36:59.792844 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0807 18:36:59.792904 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.126:40109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":95,"threads_started":1}
I20260807 18:36:59.969235 29728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:36:59.968844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:40109 (local address 127.11.14.66:58897)
0807 18:36:59.969086 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0807 18:36:59.969141 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.126:40109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":75,"threads_started":1}
I20260807 18:37:00.172623 29730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:00.172169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:40109 (local address 127.11.14.67:46699)
0807 18:37:00.172433 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0807 18:37:00.172489 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.126:40109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":89,"threads_started":1}
I20260807 18:37:00.794479 29731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:00.793945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:40109 (local address 127.11.14.65:49105)
0807 18:37:00.794317 (+ 372us) negotiation.cc:107] Waiting for socket to connect
0807 18:37:00.794381 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.126:40109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":270,"thread_start_us":80,"threads_started":1}
I20260807 18:37:00.970538 29732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:00.970093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:40109 (local address 127.11.14.66:54871)
0807 18:37:00.970388 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0807 18:37:00.970444 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.126:40109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":102,"threads_started":1}
I20260807 18:37:01.173960 29733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:01.173505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:40109 (local address 127.11.14.67:57297)
0807 18:37:01.173819 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0807 18:37:01.173873 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.126:40109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":81,"threads_started":1}
I20260807 18:37:01.795696 29734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:01.795235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:40109 (local address 127.11.14.65:35983)
0807 18:37:01.795481 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0807 18:37:01.795539 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.126:40109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":80,"threads_started":1}
I20260807 18:37:01.840431 29735 raft_consensus.cc:482] T df5fba67196f4c6892658b1580fdc44d P 8a890d9b2c9d44be9139631c5bdb2967 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:37:01.840565 29735 raft_consensus.cc:504] T df5fba67196f4c6892658b1580fdc44d P 8a890d9b2c9d44be9139631c5bdb2967 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6c9cfee63a24a18b79f2e34c4222e44" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 39057 } } peers { permanent_uuid: "8dd92e134d474cdfbb667f7435670c8d" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 39389 } } peers { permanent_uuid: "8a890d9b2c9d44be9139631c5bdb2967" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 45483 } }
I20260807 18:37:01.841089 29735 leader_election.cc:288] T df5fba67196f4c6892658b1580fdc44d P 8a890d9b2c9d44be9139631c5bdb2967 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f6c9cfee63a24a18b79f2e34c4222e44 (127.11.14.67:39057), 8dd92e134d474cdfbb667f7435670c8d (127.11.14.66:39389)
W20260807 18:37:01.841977 29288 leader_election.cc:341] T df5fba67196f4c6892658b1580fdc44d P 8a890d9b2c9d44be9139631c5bdb2967 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8dd92e134d474cdfbb667f7435670c8d (127.11.14.66:39389): Not found: Tablet not found: df5fba67196f4c6892658b1580fdc44d
W20260807 18:37:01.842124 29288 leader_election.cc:341] T df5fba67196f4c6892658b1580fdc44d P 8a890d9b2c9d44be9139631c5bdb2967 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f6c9cfee63a24a18b79f2e34c4222e44 (127.11.14.67:39057): Not found: Tablet not found: df5fba67196f4c6892658b1580fdc44d
I20260807 18:37:01.842186 29288 leader_election.cc:302] T df5fba67196f4c6892658b1580fdc44d P 8a890d9b2c9d44be9139631c5bdb2967 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a890d9b2c9d44be9139631c5bdb2967; no voters: 8dd92e134d474cdfbb667f7435670c8d, f6c9cfee63a24a18b79f2e34c4222e44
I20260807 18:37:01.842360 29735 raft_consensus.cc:2726] T df5fba67196f4c6892658b1580fdc44d P 8a890d9b2c9d44be9139631c5bdb2967 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 18:37:01.971829 29736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:01.971387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:40109 (local address 127.11.14.66:58011)
0807 18:37:01.971675 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0807 18:37:01.971728 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.126:40109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":73,"threads_started":1}
I20260807 18:37:02.175485 29737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:02.174895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:40109 (local address 127.11.14.67:35341)
0807 18:37:02.175243 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0807 18:37:02.175342 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.126:40109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":238,"thread_start_us":108,"threads_started":1}
I20260807 18:37:02.797051 29738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:02.796494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:40109 (local address 127.11.14.65:37535)
0807 18:37:02.796889 (+ 395us) negotiation.cc:107] Waiting for socket to connect
0807 18:37:02.796947 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.126:40109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":165,"threads_started":1}
I20260807 18:37:02.973069 29740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:02.972614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:40109 (local address 127.11.14.66:32789)
0807 18:37:02.972915 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0807 18:37:02.972973 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.126:40109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":72,"threads_started":1}
I20260807 18:37:03.176884 29742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:03.176285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:40109 (local address 127.11.14.67:52465)
0807 18:37:03.176691 (+ 406us) negotiation.cc:107] Waiting for socket to connect
0807 18:37:03.176777 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.126:40109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":305,"thread_start_us":78,"threads_started":1}
I20260807 18:37:03.798313 29743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:03.797838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:40109 (local address 127.11.14.65:36903)
0807 18:37:03.798154 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0807 18:37:03.798213 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.126:40109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":98,"threads_started":1}
I20260807 18:37:03.974362 29744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:03.973898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:40109 (local address 127.11.14.66:51101)
0807 18:37:03.974206 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0807 18:37:03.974258 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.126:40109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":118,"threads_started":1}
I20260807 18:37:04.178390 29745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:04.177833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:40109 (local address 127.11.14.67:40183)
0807 18:37:04.178148 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0807 18:37:04.178231 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.126:40109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":94,"threads_started":1}
I20260807 18:37:04.799854 29746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:04.799322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:40109 (local address 127.11.14.65:45937)
0807 18:37:04.799635 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0807 18:37:04.799717 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.126:40109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":99,"threads_started":1}
I20260807 18:37:04.975641 29747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:04.975132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:40109 (local address 127.11.14.66:49553)
0807 18:37:04.975420 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0807 18:37:04.975480 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.126:40109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":111,"threads_started":1}
I20260807 18:37:05.033123 29750 raft_consensus.cc:482] T df5fba67196f4c6892658b1580fdc44d P 8a890d9b2c9d44be9139631c5bdb2967 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:37:05.033290 29750 raft_consensus.cc:504] T df5fba67196f4c6892658b1580fdc44d P 8a890d9b2c9d44be9139631c5bdb2967 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6c9cfee63a24a18b79f2e34c4222e44" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 39057 } } peers { permanent_uuid: "8dd92e134d474cdfbb667f7435670c8d" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 39389 } } peers { permanent_uuid: "8a890d9b2c9d44be9139631c5bdb2967" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 45483 } }
I20260807 18:37:05.033855 29750 leader_election.cc:288] T df5fba67196f4c6892658b1580fdc44d P 8a890d9b2c9d44be9139631c5bdb2967 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f6c9cfee63a24a18b79f2e34c4222e44 (127.11.14.67:39057), 8dd92e134d474cdfbb667f7435670c8d (127.11.14.66:39389)
W20260807 18:37:05.034804 29288 leader_election.cc:341] T df5fba67196f4c6892658b1580fdc44d P 8a890d9b2c9d44be9139631c5bdb2967 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8dd92e134d474cdfbb667f7435670c8d (127.11.14.66:39389): Not found: Tablet not found: df5fba67196f4c6892658b1580fdc44d
W20260807 18:37:05.034969 29288 leader_election.cc:341] T df5fba67196f4c6892658b1580fdc44d P 8a890d9b2c9d44be9139631c5bdb2967 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f6c9cfee63a24a18b79f2e34c4222e44 (127.11.14.67:39057): Not found: Tablet not found: df5fba67196f4c6892658b1580fdc44d
I20260807 18:37:05.035036 29288 leader_election.cc:302] T df5fba67196f4c6892658b1580fdc44d P 8a890d9b2c9d44be9139631c5bdb2967 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a890d9b2c9d44be9139631c5bdb2967; no voters: 8dd92e134d474cdfbb667f7435670c8d, f6c9cfee63a24a18b79f2e34c4222e44
I20260807 18:37:05.035240 29750 raft_consensus.cc:2726] T df5fba67196f4c6892658b1580fdc44d P 8a890d9b2c9d44be9139631c5bdb2967 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 18:37:05.180078 29751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:05.179426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:40109 (local address 127.11.14.67:35543)
0807 18:37:05.179908 (+ 482us) negotiation.cc:107] Waiting for socket to connect
0807 18:37:05.179978 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.126:40109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":383,"thread_start_us":128,"threads_started":1}
I20260807 18:37:05.801401 29752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:05.800962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:40109 (local address 127.11.14.65:42215)
0807 18:37:05.801245 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0807 18:37:05.801301 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.126:40109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":96,"threads_started":1}
I20260807 18:37:05.977198 29753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:05.976526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:40109 (local address 127.11.14.66:38931)
0807 18:37:05.976921 (+ 395us) negotiation.cc:107] Waiting for socket to connect
0807 18:37:05.977030 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.126:40109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":85,"threads_started":1}
I20260807 18:37:06.181234 29754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:06.180836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:40109 (local address 127.11.14.67:36295)
0807 18:37:06.181083 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0807 18:37:06.181138 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.126:40109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":73,"threads_started":1}
I20260807 18:37:06.802749 29755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:06.802318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:40109 (local address 127.11.14.65:56469)
0807 18:37:06.802585 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0807 18:37:06.802641 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.126:40109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":77,"threads_started":1}
I20260807 18:37:06.957643 29756 raft_consensus.cc:482] T df5fba67196f4c6892658b1580fdc44d P 8a890d9b2c9d44be9139631c5bdb2967 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:37:06.957785 29756 raft_consensus.cc:504] T df5fba67196f4c6892658b1580fdc44d P 8a890d9b2c9d44be9139631c5bdb2967 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6c9cfee63a24a18b79f2e34c4222e44" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 39057 } } peers { permanent_uuid: "8dd92e134d474cdfbb667f7435670c8d" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 39389 } } peers { permanent_uuid: "8a890d9b2c9d44be9139631c5bdb2967" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 45483 } }
I20260807 18:37:06.958222 29756 leader_election.cc:288] T df5fba67196f4c6892658b1580fdc44d P 8a890d9b2c9d44be9139631c5bdb2967 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f6c9cfee63a24a18b79f2e34c4222e44 (127.11.14.67:39057), 8dd92e134d474cdfbb667f7435670c8d (127.11.14.66:39389)
W20260807 18:37:06.959143 29288 leader_election.cc:341] T df5fba67196f4c6892658b1580fdc44d P 8a890d9b2c9d44be9139631c5bdb2967 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8dd92e134d474cdfbb667f7435670c8d (127.11.14.66:39389): Not found: Tablet not found: df5fba67196f4c6892658b1580fdc44d
W20260807 18:37:06.959275 29288 leader_election.cc:341] T df5fba67196f4c6892658b1580fdc44d P 8a890d9b2c9d44be9139631c5bdb2967 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f6c9cfee63a24a18b79f2e34c4222e44 (127.11.14.67:39057): Not found: Tablet not found: df5fba67196f4c6892658b1580fdc44d
I20260807 18:37:06.959331 29288 leader_election.cc:302] T df5fba67196f4c6892658b1580fdc44d P 8a890d9b2c9d44be9139631c5bdb2967 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a890d9b2c9d44be9139631c5bdb2967; no voters: 8dd92e134d474cdfbb667f7435670c8d, f6c9cfee63a24a18b79f2e34c4222e44
I20260807 18:37:06.959534 29756 raft_consensus.cc:2726] T df5fba67196f4c6892658b1580fdc44d P 8a890d9b2c9d44be9139631c5bdb2967 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 18:37:06.978446 29757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:06.978013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:40109 (local address 127.11.14.66:54475)
0807 18:37:06.978261 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0807 18:37:06.978332 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.126:40109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":92,"threads_started":1}
I20260807 18:37:07.183295 29758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:07.182458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:40109 (local address 127.11.14.67:49345)
0807 18:37:07.182989 (+ 531us) negotiation.cc:107] Waiting for socket to connect
0807 18:37:07.183089 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.126:40109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":400,"thread_start_us":186,"threads_started":1}
I20260807 18:37:07.805068 29759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:07.804044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:40109 (local address 127.11.14.65:59995)
0807 18:37:07.804710 (+ 666us) negotiation.cc:107] Waiting for socket to connect
0807 18:37:07.804842 (+ 132us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.126:40109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":412,"thread_start_us":180,"threads_started":1}
I20260807 18:37:07.979671 29760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:07.979221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:40109 (local address 127.11.14.66:45725)
0807 18:37:07.979468 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0807 18:37:07.979521 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.126:40109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":84,"threads_started":1}
I20260807 18:37:08.184811 29761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:08.184278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:40109 (local address 127.11.14.67:34765)
0807 18:37:08.184592 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0807 18:37:08.184668 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.126:40109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":89,"threads_started":1}
I20260807 18:37:08.806613 29763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:08.806133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:40109 (local address 127.11.14.65:56773)
0807 18:37:08.806428 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0807 18:37:08.806506 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.126:40109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":82,"threads_started":1}
I20260807 18:37:08.980901 29766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:08.980470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:40109 (local address 127.11.14.66:56769)
0807 18:37:08.980741 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0807 18:37:08.980799 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.126:40109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":78,"threads_started":1}
I20260807 18:37:09.185997 29767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:09.185554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:40109 (local address 127.11.14.67:37555)
0807 18:37:09.185807 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0807 18:37:09.185874 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.126:40109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":71,"threads_started":1}
I20260807 18:37:09.808040 29768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:09.807457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:40109 (local address 127.11.14.65:53037)
0807 18:37:09.807879 (+ 422us) negotiation.cc:107] Waiting for socket to connect
0807 18:37:09.807937 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.126:40109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":347,"thread_start_us":215,"threads_started":1}
I20260807 18:37:09.982209 29769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:09.981776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:40109 (local address 127.11.14.66:54989)
0807 18:37:09.982052 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0807 18:37:09.982108 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.126:40109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":96,"threads_started":1}
I20260807 18:37:10.187304 29770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:10.186831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:40109 (local address 127.11.14.67:38863)
0807 18:37:10.187136 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0807 18:37:10.187197 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.126:40109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":85,"threads_started":1}
I20260807 18:37:10.809509 29771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:10.809047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:40109 (local address 127.11.14.65:41387)
0807 18:37:10.809322 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0807 18:37:10.809406 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.126:40109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":75,"threads_started":1}
I20260807 18:37:10.983536 29772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:10.983106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:40109 (local address 127.11.14.66:46849)
0807 18:37:10.983383 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0807 18:37:10.983435 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.126:40109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":77,"threads_started":1}
I20260807 18:37:11.178752 11321 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkz56XN/build/debug/bin/kudu with pid 29225
I20260807 18:37:11.188990 29773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:11.188379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:40109 (local address 127.11.14.67:57285)
0807 18:37:11.188764 (+ 385us) negotiation.cc:107] Waiting for socket to connect
0807 18:37:11.188843 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.126:40109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":284,"thread_start_us":81,"threads_started":1}
I20260807 18:37:11.190259 11321 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkz56XN/build/debug/bin/kudu with pid 29419
I20260807 18:37:11.200495 11321 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkz56XN/build/debug/bin/kudu with pid 29567
2026-08-07T18:37:11Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (59655 ms)
[----------] 30 tests from SecurityITest (251268 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127579900523-11321-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:37:11 dist-test-slave-6k22 krb5kdc[29776](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:37:11 dist-test-slave-6k22 krb5kdc[29776](info): set up 2 sockets
Aug 07 18:37:11 dist-test-slave-6k22 krb5kdc[29776](info): commencing operation
krb5kdc: starting...
W20260807 18:37:12.402400 11321 mini_kdc.cc:121] Time spent starting KDC: real 1.176s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127579900523-11321-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127579900523-11321-0/krb5kdc/test-user.keytab.
Aug 07 18:37:12 dist-test-slave-6k22 krb5kdc[29776](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127832, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:37:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:37:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.14.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.14.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.14.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.126.keytab.
Entry for principal kudu/127.11.14.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.126.keytab.
I20260807 18:37:12.532015 11321 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkz56XN/build/debug/bin/kudu
/tmp/dist-test-taskkz56XN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127579900523-11321-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127579900523-11321-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127579900523-11321-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127579900523-11321-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.14.126:37667
--webserver_interface=127.11.14.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.126.keytab
--principal=kudu/127.11.14.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.14.84:41245
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.14.126:37667
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127579900523-11321-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127579900523-11321-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127579900523-11321-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:37:12.645241 29791 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:37:12.645500 29791 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:37:12.645539 29791 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:37:12.650751 29791 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:37:12.650812 29791 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:37:12.650833 29791 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:37:12.650869 29791 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:37:12.650900 29791 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:37:12.656188 29791 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.14.84:41245
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127579900523-11321-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127579900523-11321-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.14.126:37667
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.126.keytab
--principal=kudu/127.11.14.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.14.126:37667
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127579900523-11321-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.14.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29791
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127579900523-11321-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:37:12.657217 29791 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:37:12.658288 29791 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 18:37:12.667448 29791 server_base.cc:1034] running on GCE node
W20260807 18:37:12.667477 29801 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:37:12.667467 29797 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:37:12.667511 29798 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:37:12.668354 29791 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:37:12.669602 29791 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:37:12.670814 29791 hybrid_clock.cc:648] HybridClock initialized: now 1786127832670785 us; error 44 us; skew 500 ppm
Aug 07 18:37:12 dist-test-slave-6k22 krb5kdc[29776](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127832, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:37:12.673717 29791 init.cc:377] Logged in from keytab as kudu/127.11.14.126@KRBTEST.COM (short username kudu)
I20260807 18:37:12.674615 29791 webserver.cc:460] Webserver started at http://127.11.14.126:40347/ using document root <none> and password file <none>
I20260807 18:37:12.675168 29791 fs_manager.cc:359] Metadata directory not provided
I20260807 18:37:12.675227 29791 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:37:12.675406 29791 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:37:12.677568 29791 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127579900523-11321-0/minicluster-data/master-0/data/instance:
uuid: "eb70e53946e94041af95a06a4a8e6daf"
format_stamp: "Formatted at 2026-08-07 18:37:12 on dist-test-slave-6k22"
server_key: "022ce338378b27169ce7cebc31735e12"
server_key_iv: "573a80725c0b5bd7607e6b016246a18f"
server_key_version: "encryptionkey@0"
I20260807 18:37:12.678176 29791 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127579900523-11321-0/minicluster-data/master-0/wal/instance:
uuid: "eb70e53946e94041af95a06a4a8e6daf"
format_stamp: "Formatted at 2026-08-07 18:37:12 on dist-test-slave-6k22"
server_key: "022ce338378b27169ce7cebc31735e12"
server_key_iv: "573a80725c0b5bd7607e6b016246a18f"
server_key_version: "encryptionkey@0"
I20260807 18:37:12.682283 29791 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 18:37:12.685072 29808 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:37:12.686445 29791 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 18:37:12.686623 29791 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127579900523-11321-0/minicluster-data/master-0/data,/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127579900523-11321-0/minicluster-data/master-0/wal
uuid: "eb70e53946e94041af95a06a4a8e6daf"
format_stamp: "Formatted at 2026-08-07 18:37:12 on dist-test-slave-6k22"
server_key: "022ce338378b27169ce7cebc31735e12"
server_key_iv: "573a80725c0b5bd7607e6b016246a18f"
server_key_version: "encryptionkey@0"
I20260807 18:37:12.686753 29791 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127579900523-11321-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127579900523-11321-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127579900523-11321-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:37:12.700573 29791 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:37:12.703516 29791 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:37:12.703802 29791 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:37:12.711647 29871 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.14.126:37667 every 8 connection(s)
I20260807 18:37:12.711652 29791 rpc_server.cc:307] RPC server started. Bound to: 127.11.14.126:37667
I20260807 18:37:12.712723 29791 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127579900523-11321-0/minicluster-data/master-0/data/info.pb
I20260807 18:37:12.715545 29872 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:37:12.718667 11321 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkz56XN/build/debug/bin/kudu as pid 29791
I20260807 18:37:12.718819 11321 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127579900523-11321-0/minicluster-data/master-0/wal/instance
I20260807 18:37:12.719106 11321 external_mini_cluster.cc:1428] Setting key 2806c9121da10d3cb6cde4961b597438
I20260807 18:37:12.721546 29872 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eb70e53946e94041af95a06a4a8e6daf: Bootstrap starting.
I20260807 18:37:12.724208 29872 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P eb70e53946e94041af95a06a4a8e6daf: Neither blocks nor log segments found. Creating new log.
I20260807 18:37:12.725006 29872 log.cc:824] T 00000000000000000000000000000000 P eb70e53946e94041af95a06a4a8e6daf: Log is configured to *not* fsync() on all Append() calls
I20260807 18:37:12.727128 29872 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eb70e53946e94041af95a06a4a8e6daf: No bootstrap required, opened a new log
Aug 07 18:37:12 dist-test-slave-6k22 krb5kdc[29776](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127832, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.14.126@KRBTEST.COM
I20260807 18:37:12.730978 29872 raft_consensus.cc:348] T 00000000000000000000000000000000 P eb70e53946e94041af95a06a4a8e6daf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb70e53946e94041af95a06a4a8e6daf" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 37667 } }
I20260807 18:37:12.731225 29872 raft_consensus.cc:374] T 00000000000000000000000000000000 P eb70e53946e94041af95a06a4a8e6daf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:37:12.731302 29872 raft_consensus.cc:729] T 00000000000000000000000000000000 P eb70e53946e94041af95a06a4a8e6daf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb70e53946e94041af95a06a4a8e6daf, State: Initialized, Role: FOLLOWER
I20260807 18:37:12.731812 29872 consensus_queue.cc:260] T 00000000000000000000000000000000 P eb70e53946e94041af95a06a4a8e6daf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb70e53946e94041af95a06a4a8e6daf" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 37667 } }
I20260807 18:37:12.731961 29872 raft_consensus.cc:388] T 00000000000000000000000000000000 P eb70e53946e94041af95a06a4a8e6daf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:37:12.732044 29872 raft_consensus.cc:482] T 00000000000000000000000000000000 P eb70e53946e94041af95a06a4a8e6daf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:37:12.732162 29872 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eb70e53946e94041af95a06a4a8e6daf [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:37:12.733057 29872 raft_consensus.cc:504] T 00000000000000000000000000000000 P eb70e53946e94041af95a06a4a8e6daf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb70e53946e94041af95a06a4a8e6daf" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 37667 } }
I20260807 18:37:12.733433 29872 leader_election.cc:302] T 00000000000000000000000000000000 P eb70e53946e94041af95a06a4a8e6daf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: eb70e53946e94041af95a06a4a8e6daf; no voters:
I20260807 18:37:12.733753 29872 leader_election.cc:288] T 00000000000000000000000000000000 P eb70e53946e94041af95a06a4a8e6daf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:37:12.733824 29877 raft_consensus.cc:2781] T 00000000000000000000000000000000 P eb70e53946e94041af95a06a4a8e6daf [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:37:12.734032 29877 raft_consensus.cc:686] T 00000000000000000000000000000000 P eb70e53946e94041af95a06a4a8e6daf [term 1 LEADER]: Becoming Leader. State: Replica: eb70e53946e94041af95a06a4a8e6daf, State: Running, Role: LEADER
I20260807 18:37:12.734388 29875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:12.720599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41980 (local address 127.11.14.126:37667)
0807 18:37:12.721051 (+ 452us) server_negotiation.cc:206] Beginning negotiation
0807 18:37:12.721068 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0807 18:37:12.721110 (+ 42us) server_negotiation.cc:407] Connection header received
0807 18:37:12.721802 (+ 692us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:37:12.721823 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:37:12.722192 (+ 369us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:37:12.722490 (+ 298us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:37:12.723490 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:12.724735 (+ 1245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:12.725793 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:12.726296 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:12.726499 (+ 203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:12.729121 (+ 2622us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:37:12.729126 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:37:12.729141 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:37:12.729169 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:37:12.730959 (+ 1790us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:37:12.731472 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:37:12.731477 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:37:12.731482 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:37:12.731655 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:37:12.731942 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:37:12.731946 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:37:12.731948 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:37:12.732494 (+ 546us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:37:12.732710 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0807 18:37:12.732988 (+ 278us) server_negotiation.cc:299] Negotiation successful
0807 18:37:12.733315 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":286,"thread_start_us":135,"threads_started":1}
I20260807 18:37:12.734409 29877 consensus_queue.cc:237] T 00000000000000000000000000000000 P eb70e53946e94041af95a06a4a8e6daf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb70e53946e94041af95a06a4a8e6daf" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 37667 } }
I20260807 18:37:12.735028 29872 sys_catalog.cc:564] T 00000000000000000000000000000000 P eb70e53946e94041af95a06a4a8e6daf [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:37:12.737061 29879 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb70e53946e94041af95a06a4a8e6daf [sys.catalog]: SysCatalogTable state changed. Reason: New leader eb70e53946e94041af95a06a4a8e6daf. Latest consensus state: current_term: 1 leader_uuid: "eb70e53946e94041af95a06a4a8e6daf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb70e53946e94041af95a06a4a8e6daf" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 37667 } } }
I20260807 18:37:12.737201 29879 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb70e53946e94041af95a06a4a8e6daf [sys.catalog]: This master's current role is: LEADER
I20260807 18:37:12.738754 29878 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb70e53946e94041af95a06a4a8e6daf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "eb70e53946e94041af95a06a4a8e6daf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb70e53946e94041af95a06a4a8e6daf" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 37667 } } }
I20260807 18:37:12.738930 29878 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb70e53946e94041af95a06a4a8e6daf [sys.catalog]: This master's current role is: LEADER
W20260807 18:37:12.740114 29892 catalog_manager.cc:1558] T 00000000000000000000000000000000 P eb70e53946e94041af95a06a4a8e6daf: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 18:37:12.740206 29892 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 18:37:12.740264 29893 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:37:12.740952 29893 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:37:12.746588 29893 catalog_manager.cc:1347] Generated new cluster ID: 53039cfbca7e4b9cb44957d8c9ededb6
I20260807 18:37:12.746659 29893 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:37:12.769610 29893 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:37:12.770643 29893 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:37:12.778520 29893 catalog_manager.cc:5868] T 00000000000000000000000000000000 P eb70e53946e94041af95a06a4a8e6daf: Generated new TSK 0
I20260807 18:37:12.779214 29893 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.14.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.14.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.14.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.65.keytab.
Entry for principal kudu/127.11.14.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.65.keytab.
I20260807 18:37:12.815274 11321 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkz56XN/build/debug/bin/kudu
/tmp/dist-test-taskkz56XN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127579900523-11321-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127579900523-11321-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127579900523-11321-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127579900523-11321-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.14.65:0
--local_ip_for_outbound_sockets=127.11.14.65
--webserver_interface=127.11.14.65
--webserver_port=0
--tserver_master_addrs=127.11.14.126:37667
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.65.keytab
--principal=kudu/127.11.14.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.14.84:41245
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127579900523-11321-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127579900523-11321-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127579900523-11321-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:37:12.944199 29900 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:37:12.944494 29900 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:37:12.944541 29900 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:37:12.949450 29900 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:37:12.949522 29900 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:37:12.949582 29900 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.14.65
I20260807 18:37:12.954892 29900 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.14.84:41245
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127579900523-11321-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127579900523-11321-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.65.keytab
--principal=kudu/127.11.14.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.14.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127579900523-11321-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.14.65
--webserver_port=0
--tserver_master_addrs=127.11.14.126:37667
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29900
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.14.65
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127579900523-11321-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:37:12.956185 29900 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:37:12.957178 29900 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:37:12.966910 29910 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:37:12.966910 29906 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:37:12.966967 29907 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:37:12.967127 29900 server_base.cc:1034] running on GCE node
I20260807 18:37:12.967649 29900 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:37:12.968289 29900 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:37:12.969455 29900 hybrid_clock.cc:648] HybridClock initialized: now 1786127832969423 us; error 43 us; skew 500 ppm
Aug 07 18:37:12 dist-test-slave-6k22 krb5kdc[29776](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127832, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:37:12.972231 29900 init.cc:377] Logged in from keytab as kudu/127.11.14.65@KRBTEST.COM (short username kudu)
I20260807 18:37:12.973083 29900 webserver.cc:460] Webserver started at http://127.11.14.65:46625/ using document root <none> and password file <none>
I20260807 18:37:12.973649 29900 fs_manager.cc:359] Metadata directory not provided
I20260807 18:37:12.973735 29900 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:37:12.973986 29900 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:37:12.976102 29900 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127579900523-11321-0/minicluster-data/ts-0/data/instance:
uuid: "144162b22242477cbc510beea0e162fc"
format_stamp: "Formatted at 2026-08-07 18:37:12 on dist-test-slave-6k22"
server_key: "0adb502d1ff972b2e203c347b5f3c98d"
server_key_iv: "060524c9fea1f33f66f2a987a8cf36e1"
server_key_version: "encryptionkey@0"
I20260807 18:37:12.976672 29900 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127579900523-11321-0/minicluster-data/ts-0/wal/instance:
uuid: "144162b22242477cbc510beea0e162fc"
format_stamp: "Formatted at 2026-08-07 18:37:12 on dist-test-slave-6k22"
server_key: "0adb502d1ff972b2e203c347b5f3c98d"
server_key_iv: "060524c9fea1f33f66f2a987a8cf36e1"
server_key_version: "encryptionkey@0"
I20260807 18:37:12.980430 29900 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260807 18:37:12.982893 29918 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:37:12.984272 29900 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260807 18:37:12.984419 29900 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127579900523-11321-0/minicluster-data/ts-0/data,/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127579900523-11321-0/minicluster-data/ts-0/wal
uuid: "144162b22242477cbc510beea0e162fc"
format_stamp: "Formatted at 2026-08-07 18:37:12 on dist-test-slave-6k22"
server_key: "0adb502d1ff972b2e203c347b5f3c98d"
server_key_iv: "060524c9fea1f33f66f2a987a8cf36e1"
server_key_version: "encryptionkey@0"
I20260807 18:37:12.984539 29900 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127579900523-11321-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127579900523-11321-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127579900523-11321-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:37:13.016494 29900 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:37:13.019870 29900 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:37:13.020036 29900 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:37:13.020591 29900 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:37:13.021399 29900 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:37:13.021448 29900 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:37:13.021488 29900 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:37:13.021510 29900 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:37:13.030416 29900 rpc_server.cc:307] RPC server started. Bound to: 127.11.14.65:40701
I20260807 18:37:13.030439 30043 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.14.65:40701 every 8 connection(s)
I20260807 18:37:13.031349 29900 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127579900523-11321-0/minicluster-data/ts-0/data/info.pb
I20260807 18:37:13.033399 11321 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkz56XN/build/debug/bin/kudu as pid 29900
I20260807 18:37:13.033537 11321 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127579900523-11321-0/minicluster-data/ts-0/wal/instance
I20260807 18:37:13.033843 11321 external_mini_cluster.cc:1428] Setting key 20f17a0735d35898c829e96d9fd9e3a7
Aug 07 18:37:13 dist-test-slave-6k22 krb5kdc[29776](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127832, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.65@KRBTEST.COM for kudu/127.11.14.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.11.14.66@KRBTEST.COM; defaulting to no policy
I20260807 18:37:13.046047 29875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:13.033198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.65:56333 (local address 127.11.14.126:37667)
0807 18:37:13.033366 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0807 18:37:13.033372 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:37:13.034569 (+ 1197us) server_negotiation.cc:407] Connection header received
0807 18:37:13.035852 (+ 1283us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:37:13.035855 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:37:13.035897 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:37:13.035983 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:37:13.037295 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:13.037832 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:13.038989 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:13.039223 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:13.039335 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:13.042321 (+ 2986us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:37:13.042325 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:37:13.042328 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:37:13.042358 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:37:13.043896 (+ 1538us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:37:13.044437 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:37:13.044441 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:37:13.044443 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:37:13.044528 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:37:13.044906 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:37:13.044911 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:37:13.044913 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:37:13.045120 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:37:13.045230 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0807 18:37:13.045729 (+ 499us) server_negotiation.cc:299] Negotiation successful
0807 18:37:13.045864 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260807 18:37:13.046892 30046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:13.033639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:37667 (local address 127.11.14.65:56333)
0807 18:37:13.034403 (+ 764us) negotiation.cc:107] Waiting for socket to connect
0807 18:37:13.034469 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0807 18:37:13.035596 (+ 1127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:37:13.036179 (+ 583us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:37:13.036186 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:37:13.036669 (+ 483us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:37:13.037154 (+ 485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:37:13.037168 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:37:13.037958 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:37:13.037961 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:37:13.038862 (+ 901us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:37:13.038872 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:37:13.039382 (+ 510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:37:13.039386 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:37:13.039615 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:13.040380 (+ 765us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:37:13.040397 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:37:13.042135 (+ 1738us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:37:13.044009 (+ 1874us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:37:13.044024 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:37:13.044039 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:37:13.044307 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:37:13.044623 (+ 316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:37:13.044626 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:37:13.044628 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:37:13.044808 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:37:13.045231 (+ 423us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:37:13.045239 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:37:13.045492 (+ 253us) client_negotiation.cc:769] Sending connection context
0807 18:37:13.045686 (+ 194us) client_negotiation.cc:240] Negotiation successful
0807 18:37:13.046042 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":539,"thread_start_us":119,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.14.66@KRBTEST.COM" created.
I20260807 18:37:13.048209 30044 heartbeater.cc:347] Connected to a master server at 127.11.14.126:37667
I20260807 18:37:13.048482 30044 heartbeater.cc:464] Registering TS with master...
I20260807 18:37:13.048993 30044 heartbeater.cc:511] Master 127.11.14.126:37667 requested a full tablet report, sending...
I20260807 18:37:13.051141 29830 ts_manager.cc:194] Registered new tserver with Master: 144162b22242477cbc510beea0e162fc (127.11.14.65:40701)
I20260807 18:37:13.052341 29830 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.14.65@KRBTEST.COM'} at 127.11.14.65:56333
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.14.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.66.keytab.
Entry for principal kudu/127.11.14.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.66.keytab.
I20260807 18:37:13.064504 11321 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkz56XN/build/debug/bin/kudu
/tmp/dist-test-taskkz56XN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127579900523-11321-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127579900523-11321-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127579900523-11321-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127579900523-11321-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.14.66:0
--local_ip_for_outbound_sockets=127.11.14.66
--webserver_interface=127.11.14.66
--webserver_port=0
--tserver_master_addrs=127.11.14.126:37667
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.66.keytab
--principal=kudu/127.11.14.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.14.84:41245
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127579900523-11321-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127579900523-11321-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127579900523-11321-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:37:13.178213 30050 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:37:13.178526 30050 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:37:13.178601 30050 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:37:13.182628 30050 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:37:13.182708 30050 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:37:13.182796 30050 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.14.66
I20260807 18:37:13.188230 30050 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.14.84:41245
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127579900523-11321-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127579900523-11321-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.66.keytab
--principal=kudu/127.11.14.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.14.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127579900523-11321-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.14.66
--webserver_port=0
--tserver_master_addrs=127.11.14.126:37667
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30050
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.14.66
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127579900523-11321-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:37:13.189342 30050 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:37:13.190418 30050 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:37:13.200047 30056 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:37:13.200053 30057 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:37:13.200078 30059 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:37:13.200340 30050 server_base.cc:1034] running on GCE node
I20260807 18:37:13.200839 30050 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:37:13.201521 30050 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:37:13.202702 30050 hybrid_clock.cc:648] HybridClock initialized: now 1786127833202686 us; error 29 us; skew 500 ppm
Aug 07 18:37:13 dist-test-slave-6k22 krb5kdc[29776](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127833, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:37:13.205457 30050 init.cc:377] Logged in from keytab as kudu/127.11.14.66@KRBTEST.COM (short username kudu)
I20260807 18:37:13.206365 30050 webserver.cc:460] Webserver started at http://127.11.14.66:42707/ using document root <none> and password file <none>
I20260807 18:37:13.206940 30050 fs_manager.cc:359] Metadata directory not provided
I20260807 18:37:13.207031 30050 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:37:13.207261 30050 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:37:13.209334 30050 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127579900523-11321-0/minicluster-data/ts-1/data/instance:
uuid: "7eea4d71f76945ab9b338c8ab5b2442b"
format_stamp: "Formatted at 2026-08-07 18:37:13 on dist-test-slave-6k22"
server_key: "9bd466e41bdf5dae247e0148993b46d7"
server_key_iv: "17821315a89205663c3e91e164ced799"
server_key_version: "encryptionkey@0"
I20260807 18:37:13.209911 30050 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127579900523-11321-0/minicluster-data/ts-1/wal/instance:
uuid: "7eea4d71f76945ab9b338c8ab5b2442b"
format_stamp: "Formatted at 2026-08-07 18:37:13 on dist-test-slave-6k22"
server_key: "9bd466e41bdf5dae247e0148993b46d7"
server_key_iv: "17821315a89205663c3e91e164ced799"
server_key_version: "encryptionkey@0"
I20260807 18:37:13.213677 30050 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260807 18:37:13.216084 30066 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:37:13.217260 30050 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 18:37:13.217392 30050 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127579900523-11321-0/minicluster-data/ts-1/data,/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127579900523-11321-0/minicluster-data/ts-1/wal
uuid: "7eea4d71f76945ab9b338c8ab5b2442b"
format_stamp: "Formatted at 2026-08-07 18:37:13 on dist-test-slave-6k22"
server_key: "9bd466e41bdf5dae247e0148993b46d7"
server_key_iv: "17821315a89205663c3e91e164ced799"
server_key_version: "encryptionkey@0"
I20260807 18:37:13.217499 30050 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127579900523-11321-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127579900523-11321-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127579900523-11321-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:37:13.230876 30050 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:37:13.233744 30050 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:37:13.233960 30050 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:37:13.234555 30050 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:37:13.235404 30050 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:37:13.235476 30050 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:37:13.235574 30050 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:37:13.235616 30050 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:37:13.244782 30050 rpc_server.cc:307] RPC server started. Bound to: 127.11.14.66:37897
I20260807 18:37:13.244802 30193 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.14.66:37897 every 8 connection(s)
I20260807 18:37:13.245632 30050 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127579900523-11321-0/minicluster-data/ts-1/data/info.pb
I20260807 18:37:13.251197 11321 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkz56XN/build/debug/bin/kudu as pid 30050
I20260807 18:37:13.251320 11321 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127579900523-11321-0/minicluster-data/ts-1/wal/instance
I20260807 18:37:13.251631 11321 external_mini_cluster.cc:1428] Setting key b1fe4cce31f577840e542b62b3116cfd
Aug 07 18:37:13 dist-test-slave-6k22 krb5kdc[29776](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127833, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.66@KRBTEST.COM for kudu/127.11.14.126@KRBTEST.COM
I20260807 18:37:13.259631 29875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:13.247464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.66:37553 (local address 127.11.14.126:37667)
0807 18:37:13.247626 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0807 18:37:13.247629 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 18:37:13.248659 (+ 1030us) server_negotiation.cc:407] Connection header received
0807 18:37:13.249547 (+ 888us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:37:13.249551 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:37:13.249610 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:37:13.249705 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:37:13.250957 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:13.251770 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:13.253111 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:13.253424 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:13.253545 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:13.256097 (+ 2552us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:37:13.256100 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:37:13.256102 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:37:13.256124 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:37:13.257529 (+ 1405us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:37:13.258036 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:37:13.258040 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:37:13.258042 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:37:13.258122 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:37:13.258472 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:37:13.258476 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:37:13.258478 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:37:13.258667 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:37:13.258781 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0807 18:37:13.259303 (+ 522us) server_negotiation.cc:299] Negotiation successful
0807 18:37:13.259437 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260807 18:37:13.260479 30196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:13.247876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:37667 (local address 127.11.14.66:37553)
0807 18:37:13.248517 (+ 641us) negotiation.cc:107] Waiting for socket to connect
0807 18:37:13.248556 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0807 18:37:13.249339 (+ 783us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:37:13.249878 (+ 539us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:37:13.249885 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:37:13.250342 (+ 457us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:37:13.250819 (+ 477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:37:13.250832 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:37:13.251899 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:37:13.251903 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:37:13.252985 (+ 1082us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:37:13.253000 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:37:13.253568 (+ 568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:37:13.253571 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:37:13.253718 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:13.254352 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:37:13.254369 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:37:13.255927 (+ 1558us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:37:13.257647 (+ 1720us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:37:13.257653 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:37:13.257669 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:37:13.257935 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:37:13.258214 (+ 279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:37:13.258217 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:37:13.258219 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:37:13.258377 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:37:13.258806 (+ 429us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:37:13.258815 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:37:13.259067 (+ 252us) client_negotiation.cc:769] Sending connection context
0807 18:37:13.259260 (+ 193us) client_negotiation.cc:240] Negotiation successful
0807 18:37:13.259631 (+ 371us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":456,"thread_start_us":110,"threads_started":1}
I20260807 18:37:13.261693 30194 heartbeater.cc:347] Connected to a master server at 127.11.14.126:37667
I20260807 18:37:13.261957 30194 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.11.14.67@KRBTEST.COM; defaulting to no policy
I20260807 18:37:13.262498 30194 heartbeater.cc:511] Master 127.11.14.126:37667 requested a full tablet report, sending...
I20260807 18:37:13.263644 29830 ts_manager.cc:194] Registered new tserver with Master: 7eea4d71f76945ab9b338c8ab5b2442b (127.11.14.66:37897)
I20260807 18:37:13.264343 29830 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.14.66@KRBTEST.COM'} at 127.11.14.66:37553
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.14.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.14.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.67.keytab.
Entry for principal kudu/127.11.14.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.67.keytab.
I20260807 18:37:13.282819 11321 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkz56XN/build/debug/bin/kudu
/tmp/dist-test-taskkz56XN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127579900523-11321-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127579900523-11321-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127579900523-11321-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127579900523-11321-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.14.67:0
--local_ip_for_outbound_sockets=127.11.14.67
--webserver_interface=127.11.14.67
--webserver_port=0
--tserver_master_addrs=127.11.14.126:37667
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.67.keytab
--principal=kudu/127.11.14.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.14.84:41245
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127579900523-11321-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127579900523-11321-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127579900523-11321-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:37:13.398521 30199 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:37:13.398770 30199 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:37:13.398846 30199 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:37:13.402966 30199 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:37:13.403030 30199 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:37:13.403144 30199 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.14.67
I20260807 18:37:13.408506 30199 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.14.84:41245
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127579900523-11321-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127579900523-11321-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.67.keytab
--principal=kudu/127.11.14.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.14.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127579900523-11321-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.14.67
--webserver_port=0
--tserver_master_addrs=127.11.14.126:37667
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30199
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.14.67
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127579900523-11321-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:37:13.409623 30199 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:37:13.410676 30199 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:37:13.420493 30207 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:37:13.420516 30209 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:37:13.420532 30204 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:37:13.421005 30199 server_base.cc:1034] running on GCE node
I20260807 18:37:13.421451 30199 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:37:13.422062 30199 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:37:13.423243 30199 hybrid_clock.cc:648] HybridClock initialized: now 1786127833423229 us; error 35 us; skew 500 ppm
Aug 07 18:37:13 dist-test-slave-6k22 krb5kdc[29776](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127833, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:37:13.425993 30199 init.cc:377] Logged in from keytab as kudu/127.11.14.67@KRBTEST.COM (short username kudu)
I20260807 18:37:13.426796 30199 webserver.cc:460] Webserver started at http://127.11.14.67:45451/ using document root <none> and password file <none>
I20260807 18:37:13.427320 30199 fs_manager.cc:359] Metadata directory not provided
I20260807 18:37:13.427379 30199 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:37:13.427546 30199 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:37:13.429600 30199 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127579900523-11321-0/minicluster-data/ts-2/data/instance:
uuid: "8e4f59ea9e2e459085a2add77c56362c"
format_stamp: "Formatted at 2026-08-07 18:37:13 on dist-test-slave-6k22"
server_key: "59110793048fea4afcb135da9256ec5d"
server_key_iv: "cd3f0e6b1976ddb32ca3a074a22079e3"
server_key_version: "encryptionkey@0"
I20260807 18:37:13.430094 30199 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127579900523-11321-0/minicluster-data/ts-2/wal/instance:
uuid: "8e4f59ea9e2e459085a2add77c56362c"
format_stamp: "Formatted at 2026-08-07 18:37:13 on dist-test-slave-6k22"
server_key: "59110793048fea4afcb135da9256ec5d"
server_key_iv: "cd3f0e6b1976ddb32ca3a074a22079e3"
server_key_version: "encryptionkey@0"
I20260807 18:37:13.433723 30199 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.003s
I20260807 18:37:13.436244 30217 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:37:13.437476 30199 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 18:37:13.437589 30199 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127579900523-11321-0/minicluster-data/ts-2/data,/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127579900523-11321-0/minicluster-data/ts-2/wal
uuid: "8e4f59ea9e2e459085a2add77c56362c"
format_stamp: "Formatted at 2026-08-07 18:37:13 on dist-test-slave-6k22"
server_key: "59110793048fea4afcb135da9256ec5d"
server_key_iv: "cd3f0e6b1976ddb32ca3a074a22079e3"
server_key_version: "encryptionkey@0"
I20260807 18:37:13.437705 30199 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127579900523-11321-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127579900523-11321-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127579900523-11321-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:37:13.461134 30199 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:37:13.464205 30199 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:37:13.464427 30199 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:37:13.465070 30199 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:37:13.466015 30199 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:37:13.466102 30199 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:37:13.466179 30199 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:37:13.466231 30199 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:37:13.476081 30199 rpc_server.cc:307] RPC server started. Bound to: 127.11.14.67:45015
I20260807 18:37:13.476114 30342 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.14.67:45015 every 8 connection(s)
I20260807 18:37:13.477090 30199 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127579900523-11321-0/minicluster-data/ts-2/data/info.pb
I20260807 18:37:13.480430 11321 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkz56XN/build/debug/bin/kudu as pid 30199
I20260807 18:37:13.480581 11321 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127579900523-11321-0/minicluster-data/ts-2/wal/instance
I20260807 18:37:13.480911 11321 external_mini_cluster.cc:1428] Setting key 733b2db92ea5c060d69b1ff0b87cc677
Aug 07 18:37:13 dist-test-slave-6k22 krb5kdc[29776](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127833, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.67@KRBTEST.COM for kudu/127.11.14.126@KRBTEST.COM
I20260807 18:37:13.490808 29875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:13.479060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.67:39307 (local address 127.11.14.126:37667)
0807 18:37:13.479204 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0807 18:37:13.479208 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:37:13.480371 (+ 1163us) server_negotiation.cc:407] Connection header received
0807 18:37:13.481563 (+ 1192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:37:13.481566 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:37:13.481622 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:37:13.481707 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:37:13.483107 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:13.483703 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:13.484802 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:13.485011 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:13.485083 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:13.487476 (+ 2393us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:37:13.487479 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:37:13.487481 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:37:13.487505 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:37:13.488882 (+ 1377us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:37:13.489359 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:37:13.489362 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:37:13.489363 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:37:13.489454 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:37:13.489729 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:37:13.489731 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:37:13.489732 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:37:13.490011 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:37:13.490092 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0807 18:37:13.490569 (+ 477us) server_negotiation.cc:299] Negotiation successful
0807 18:37:13.490675 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260807 18:37:13.491673 30345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:13.479434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:37667 (local address 127.11.14.67:39307)
0807 18:37:13.480175 (+ 741us) negotiation.cc:107] Waiting for socket to connect
0807 18:37:13.480237 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0807 18:37:13.481345 (+ 1108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:37:13.481928 (+ 583us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:37:13.481937 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:37:13.482424 (+ 487us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:37:13.482948 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:37:13.482962 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:37:13.483809 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:37:13.483812 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:37:13.484688 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:37:13.484697 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:37:13.485134 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:37:13.485137 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:37:13.485277 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:13.485817 (+ 540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:37:13.485833 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:37:13.487274 (+ 1441us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:37:13.488985 (+ 1711us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:37:13.488989 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:37:13.489001 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:37:13.489273 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:37:13.489519 (+ 246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:37:13.489521 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:37:13.489523 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:37:13.489654 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:37:13.490102 (+ 448us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:37:13.490107 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:37:13.490360 (+ 253us) client_negotiation.cc:769] Sending connection context
0807 18:37:13.490516 (+ 156us) client_negotiation.cc:240] Negotiation successful
0807 18:37:13.490765 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":504,"thread_start_us":138,"threads_started":1}
I20260807 18:37:13.492745 30343 heartbeater.cc:347] Connected to a master server at 127.11.14.126:37667
I20260807 18:37:13.493006 30343 heartbeater.cc:464] Registering TS with master...
I20260807 18:37:13.493578 30343 heartbeater.cc:511] Master 127.11.14.126:37667 requested a full tablet report, sending...
I20260807 18:37:13.494710 29829 ts_manager.cc:194] Registered new tserver with Master: 8e4f59ea9e2e459085a2add77c56362c (127.11.14.67:45015)
I20260807 18:37:13.495334 29829 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.14.67@KRBTEST.COM'} at 127.11.14.67:39307
I20260807 18:37:13.506705 11321 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 18:37:13.515492 29875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:13.508764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41988 (local address 127.11.14.126:37667)
0807 18:37:13.508888 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0807 18:37:13.508891 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 18:37:13.509029 (+ 138us) server_negotiation.cc:407] Connection header received
0807 18:37:13.509130 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:37:13.509132 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:37:13.509171 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:37:13.509239 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:37:13.509906 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:13.510413 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:13.511192 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:13.511402 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:13.511474 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:13.512864 (+ 1390us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:37:13.512867 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:37:13.512869 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:37:13.512890 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:37:13.514100 (+ 1210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:37:13.514471 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:37:13.514473 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:37:13.514474 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:37:13.514556 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:37:13.514771 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:37:13.514773 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:37:13.514774 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:37:13.514977 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:37:13.515086 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0807 18:37:13.515279 (+ 193us) server_negotiation.cc:299] Negotiation successful
0807 18:37:13.515372 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260807 18:37:13.518373 29829 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41988:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 18:37:13.521646 29829 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 18:37:13.536096 30361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:13.531100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51826 (local address 127.11.14.67:45015)
0807 18:37:13.531522 (+ 422us) server_negotiation.cc:206] Beginning negotiation
0807 18:37:13.531535 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 18:37:13.531579 (+ 44us) server_negotiation.cc:407] Connection header received
0807 18:37:13.531661 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:37:13.531666 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:37:13.531853 (+ 187us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:37:13.531998 (+ 145us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:37:13.532251 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:13.533078 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:13.534541 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:13.535362 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:13.535463 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:13.535644 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0807 18:37:13.535843 (+ 199us) server_negotiation.cc:299] Negotiation successful
0807 18:37:13.535955 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":321,"thread_start_us":185,"threads_started":1}
I20260807 18:37:13.536087 30359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:13.529852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.67:45015 (local address 127.0.0.1:51826)
0807 18:37:13.530746 (+ 894us) negotiation.cc:107] Waiting for socket to connect
0807 18:37:13.530759 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 18:37:13.531042 (+ 283us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:37:13.532004 (+ 962us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:37:13.532007 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:37:13.532026 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:37:13.532137 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:37:13.532143 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:37:13.533208 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:37:13.533212 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:37:13.534401 (+ 1189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:37:13.534411 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:37:13.535465 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:37:13.535468 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:37:13.535617 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:13.535681 (+ 64us) client_negotiation.cc:769] Sending connection context
0807 18:37:13.535802 (+ 121us) client_negotiation.cc:240] Negotiation successful
0807 18:37:13.535947 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":832,"mutex_wait_us":110,"thread_start_us":98,"threads_started":1}
I20260807 18:37:13.536126 30358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:13.529918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38012 (local address 127.11.14.65:40701)
0807 18:37:13.530541 (+ 623us) server_negotiation.cc:206] Beginning negotiation
0807 18:37:13.530561 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0807 18:37:13.530590 (+ 29us) server_negotiation.cc:407] Connection header received
0807 18:37:13.530677 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:37:13.530684 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:37:13.530918 (+ 234us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:37:13.531088 (+ 170us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:37:13.531661 (+ 573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:13.532748 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:13.534508 (+ 1760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:13.535176 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:13.535242 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:13.535376 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0807 18:37:13.535873 (+ 497us) server_negotiation.cc:299] Negotiation successful
0807 18:37:13.535981 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":501,"thread_start_us":83,"threads_started":1}
I20260807 18:37:13.536629 30357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:13.529768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.65:40701 (local address 127.0.0.1:38012)
0807 18:37:13.530220 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0807 18:37:13.530268 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0807 18:37:13.530428 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:37:13.531319 (+ 891us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:37:13.531326 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:37:13.531358 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:37:13.531515 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:37:13.531522 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:37:13.532915 (+ 1393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:37:13.532918 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:37:13.534357 (+ 1439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:37:13.534368 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:37:13.535525 (+ 1157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:37:13.535528 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:37:13.535648 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:13.535700 (+ 52us) client_negotiation.cc:769] Sending connection context
0807 18:37:13.536379 (+ 679us) client_negotiation.cc:240] Negotiation successful
0807 18:37:13.536456 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":281,"mutex_wait_us":75,"thread_start_us":89,"threads_started":1}
I20260807 18:37:13.537590 30356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:13.530510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.66:37897 (local address 127.0.0.1:50438)
0807 18:37:13.531366 (+ 856us) negotiation.cc:107] Waiting for socket to connect
0807 18:37:13.531380 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 18:37:13.531530 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:37:13.532189 (+ 659us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:37:13.532212 (+ 23us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:37:13.532233 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:37:13.532363 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:37:13.532370 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:37:13.533794 (+ 1424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:37:13.533797 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:37:13.535264 (+ 1467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:37:13.535276 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:37:13.537172 (+ 1896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:37:13.537176 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:37:13.537271 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:13.537293 (+ 22us) client_negotiation.cc:769] Sending connection context
0807 18:37:13.537403 (+ 110us) client_negotiation.cc:240] Negotiation successful
0807 18:37:13.537469 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":781,"mutex_wait_us":77,"thread_start_us":107,"threads_started":1}
I20260807 18:37:13.537701 30360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:13.530689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50438 (local address 127.11.14.66:37897)
0807 18:37:13.531296 (+ 607us) server_negotiation.cc:206] Beginning negotiation
0807 18:37:13.531329 (+ 33us) server_negotiation.cc:399] Waiting for connection header
0807 18:37:13.531460 (+ 131us) server_negotiation.cc:407] Connection header received
0807 18:37:13.531706 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:37:13.531711 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:37:13.531870 (+ 159us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:37:13.531985 (+ 115us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:37:13.532511 (+ 526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:13.533654 (+ 1143us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:13.535410 (+ 1756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:13.536311 (+ 901us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:13.536423 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:13.536566 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0807 18:37:13.537446 (+ 880us) server_negotiation.cc:299] Negotiation successful
0807 18:37:13.537548 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":494,"thread_start_us":126,"threads_started":1}
I20260807 18:37:13.539441 30272 tablet_service.cc:1467] Processing CreateTablet for tablet 4f82ae4ee88e4e5a9219e20422293658 (DEFAULT_TABLE table=test-table [id=7707f9f41e4249808d8be6bbb7a5ee64]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:37:13.540697 30272 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4f82ae4ee88e4e5a9219e20422293658. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:37:13.540403 30122 tablet_service.cc:1467] Processing CreateTablet for tablet 4f82ae4ee88e4e5a9219e20422293658 (DEFAULT_TABLE table=test-table [id=7707f9f41e4249808d8be6bbb7a5ee64]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:37:13.541452 30122 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4f82ae4ee88e4e5a9219e20422293658. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:37:13.541354 29969 tablet_service.cc:1467] Processing CreateTablet for tablet 4f82ae4ee88e4e5a9219e20422293658 (DEFAULT_TABLE table=test-table [id=7707f9f41e4249808d8be6bbb7a5ee64]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:37:13.542230 29969 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4f82ae4ee88e4e5a9219e20422293658. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:37:13.548609 30362 tablet_bootstrap.cc:492] T 4f82ae4ee88e4e5a9219e20422293658 P 144162b22242477cbc510beea0e162fc: Bootstrap starting.
I20260807 18:37:13.549070 30364 tablet_bootstrap.cc:492] T 4f82ae4ee88e4e5a9219e20422293658 P 8e4f59ea9e2e459085a2add77c56362c: Bootstrap starting.
I20260807 18:37:13.549602 30363 tablet_bootstrap.cc:492] T 4f82ae4ee88e4e5a9219e20422293658 P 7eea4d71f76945ab9b338c8ab5b2442b: Bootstrap starting.
I20260807 18:37:13.551535 30364 tablet_bootstrap.cc:654] T 4f82ae4ee88e4e5a9219e20422293658 P 8e4f59ea9e2e459085a2add77c56362c: Neither blocks nor log segments found. Creating new log.
I20260807 18:37:13.551688 30363 tablet_bootstrap.cc:654] T 4f82ae4ee88e4e5a9219e20422293658 P 7eea4d71f76945ab9b338c8ab5b2442b: Neither blocks nor log segments found. Creating new log.
I20260807 18:37:13.553077 30363 log.cc:824] T 4f82ae4ee88e4e5a9219e20422293658 P 7eea4d71f76945ab9b338c8ab5b2442b: Log is configured to *not* fsync() on all Append() calls
I20260807 18:37:13.553077 30364 log.cc:824] T 4f82ae4ee88e4e5a9219e20422293658 P 8e4f59ea9e2e459085a2add77c56362c: Log is configured to *not* fsync() on all Append() calls
I20260807 18:37:13.553678 30362 tablet_bootstrap.cc:654] T 4f82ae4ee88e4e5a9219e20422293658 P 144162b22242477cbc510beea0e162fc: Neither blocks nor log segments found. Creating new log.
I20260807 18:37:13.554397 30362 log.cc:824] T 4f82ae4ee88e4e5a9219e20422293658 P 144162b22242477cbc510beea0e162fc: Log is configured to *not* fsync() on all Append() calls
I20260807 18:37:13.555199 30363 tablet_bootstrap.cc:492] T 4f82ae4ee88e4e5a9219e20422293658 P 7eea4d71f76945ab9b338c8ab5b2442b: No bootstrap required, opened a new log
I20260807 18:37:13.555418 30363 ts_tablet_manager.cc:1397] T 4f82ae4ee88e4e5a9219e20422293658 P 7eea4d71f76945ab9b338c8ab5b2442b: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.005s
I20260807 18:37:13.555904 30364 tablet_bootstrap.cc:492] T 4f82ae4ee88e4e5a9219e20422293658 P 8e4f59ea9e2e459085a2add77c56362c: No bootstrap required, opened a new log
I20260807 18:37:13.556094 30364 ts_tablet_manager.cc:1397] T 4f82ae4ee88e4e5a9219e20422293658 P 8e4f59ea9e2e459085a2add77c56362c: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.004s
I20260807 18:37:13.556207 30362 tablet_bootstrap.cc:492] T 4f82ae4ee88e4e5a9219e20422293658 P 144162b22242477cbc510beea0e162fc: No bootstrap required, opened a new log
I20260807 18:37:13.556378 30362 ts_tablet_manager.cc:1397] T 4f82ae4ee88e4e5a9219e20422293658 P 144162b22242477cbc510beea0e162fc: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.000s
I20260807 18:37:13.559548 30363 raft_consensus.cc:348] T 4f82ae4ee88e4e5a9219e20422293658 P 7eea4d71f76945ab9b338c8ab5b2442b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "144162b22242477cbc510beea0e162fc" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 40701 } } peers { permanent_uuid: "8e4f59ea9e2e459085a2add77c56362c" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 45015 } } peers { permanent_uuid: "7eea4d71f76945ab9b338c8ab5b2442b" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 37897 } }
I20260807 18:37:13.559883 30363 raft_consensus.cc:374] T 4f82ae4ee88e4e5a9219e20422293658 P 7eea4d71f76945ab9b338c8ab5b2442b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:37:13.559954 30363 raft_consensus.cc:729] T 4f82ae4ee88e4e5a9219e20422293658 P 7eea4d71f76945ab9b338c8ab5b2442b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7eea4d71f76945ab9b338c8ab5b2442b, State: Initialized, Role: FOLLOWER
I20260807 18:37:13.559967 30364 raft_consensus.cc:348] T 4f82ae4ee88e4e5a9219e20422293658 P 8e4f59ea9e2e459085a2add77c56362c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "144162b22242477cbc510beea0e162fc" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 40701 } } peers { permanent_uuid: "8e4f59ea9e2e459085a2add77c56362c" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 45015 } } peers { permanent_uuid: "7eea4d71f76945ab9b338c8ab5b2442b" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 37897 } }
I20260807 18:37:13.560001 30362 raft_consensus.cc:348] T 4f82ae4ee88e4e5a9219e20422293658 P 144162b22242477cbc510beea0e162fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "144162b22242477cbc510beea0e162fc" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 40701 } } peers { permanent_uuid: "8e4f59ea9e2e459085a2add77c56362c" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 45015 } } peers { permanent_uuid: "7eea4d71f76945ab9b338c8ab5b2442b" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 37897 } }
I20260807 18:37:13.560225 30364 raft_consensus.cc:374] T 4f82ae4ee88e4e5a9219e20422293658 P 8e4f59ea9e2e459085a2add77c56362c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:37:13.560254 30362 raft_consensus.cc:374] T 4f82ae4ee88e4e5a9219e20422293658 P 144162b22242477cbc510beea0e162fc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:37:13.560273 30364 raft_consensus.cc:729] T 4f82ae4ee88e4e5a9219e20422293658 P 8e4f59ea9e2e459085a2add77c56362c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e4f59ea9e2e459085a2add77c56362c, State: Initialized, Role: FOLLOWER
I20260807 18:37:13.560315 30362 raft_consensus.cc:729] T 4f82ae4ee88e4e5a9219e20422293658 P 144162b22242477cbc510beea0e162fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 144162b22242477cbc510beea0e162fc, State: Initialized, Role: FOLLOWER
I20260807 18:37:13.560463 30363 consensus_queue.cc:260] T 4f82ae4ee88e4e5a9219e20422293658 P 7eea4d71f76945ab9b338c8ab5b2442b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "144162b22242477cbc510beea0e162fc" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 40701 } } peers { permanent_uuid: "8e4f59ea9e2e459085a2add77c56362c" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 45015 } } peers { permanent_uuid: "7eea4d71f76945ab9b338c8ab5b2442b" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 37897 } }
I20260807 18:37:13.560668 30364 consensus_queue.cc:260] T 4f82ae4ee88e4e5a9219e20422293658 P 8e4f59ea9e2e459085a2add77c56362c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "144162b22242477cbc510beea0e162fc" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 40701 } } peers { permanent_uuid: "8e4f59ea9e2e459085a2add77c56362c" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 45015 } } peers { permanent_uuid: "7eea4d71f76945ab9b338c8ab5b2442b" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 37897 } }
I20260807 18:37:13.560813 30362 consensus_queue.cc:260] T 4f82ae4ee88e4e5a9219e20422293658 P 144162b22242477cbc510beea0e162fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "144162b22242477cbc510beea0e162fc" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 40701 } } peers { permanent_uuid: "8e4f59ea9e2e459085a2add77c56362c" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 45015 } } peers { permanent_uuid: "7eea4d71f76945ab9b338c8ab5b2442b" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 37897 } }
I20260807 18:37:13.561286 30194 heartbeater.cc:503] Master 127.11.14.126:37667 was elected leader, sending a full tablet report...
I20260807 18:37:13.561553 30363 ts_tablet_manager.cc:1428] T 4f82ae4ee88e4e5a9219e20422293658 P 7eea4d71f76945ab9b338c8ab5b2442b: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260807 18:37:13.561553 30364 ts_tablet_manager.cc:1428] T 4f82ae4ee88e4e5a9219e20422293658 P 8e4f59ea9e2e459085a2add77c56362c: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260807 18:37:13.561750 30362 ts_tablet_manager.cc:1428] T 4f82ae4ee88e4e5a9219e20422293658 P 144162b22242477cbc510beea0e162fc: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260807 18:37:13.561895 30343 heartbeater.cc:503] Master 127.11.14.126:37667 was elected leader, sending a full tablet report...
I20260807 18:37:13.562055 30044 heartbeater.cc:503] Master 127.11.14.126:37667 was elected leader, sending a full tablet report...
I20260807 18:37:13.719322 30371 raft_consensus.cc:482] T 4f82ae4ee88e4e5a9219e20422293658 P 8e4f59ea9e2e459085a2add77c56362c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:37:13.719605 30371 raft_consensus.cc:504] T 4f82ae4ee88e4e5a9219e20422293658 P 8e4f59ea9e2e459085a2add77c56362c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "144162b22242477cbc510beea0e162fc" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 40701 } } peers { permanent_uuid: "8e4f59ea9e2e459085a2add77c56362c" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 45015 } } peers { permanent_uuid: "7eea4d71f76945ab9b338c8ab5b2442b" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 37897 } }
I20260807 18:37:13.720871 30371 leader_election.cc:288] T 4f82ae4ee88e4e5a9219e20422293658 P 8e4f59ea9e2e459085a2add77c56362c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 144162b22242477cbc510beea0e162fc (127.11.14.65:40701), 7eea4d71f76945ab9b338c8ab5b2442b (127.11.14.66:37897)
I20260807 18:37:13.725589 30373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:13.721107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.65:40701 (local address 127.11.14.67:45335)
0807 18:37:13.721477 (+ 370us) negotiation.cc:107] Waiting for socket to connect
0807 18:37:13.721492 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 18:37:13.721619 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:37:13.722000 (+ 381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:37:13.722003 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:37:13.722045 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:37:13.722296 (+ 251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:37:13.722302 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:37:13.723357 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:37:13.723362 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:37:13.724433 (+ 1071us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:37:13.724442 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:37:13.725239 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:37:13.725244 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:37:13.725335 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:13.725357 (+ 22us) client_negotiation.cc:769] Sending connection context
0807 18:37:13.725416 (+ 59us) client_negotiation.cc:240] Negotiation successful
0807 18:37:13.725470 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":298,"mutex_wait_us":66,"spinlock_wait_cycles":8192}
I20260807 18:37:13.725651 30358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:13.721262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.67:45335 (local address 127.11.14.65:40701)
0807 18:37:13.721407 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0807 18:37:13.721412 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:37:13.721569 (+ 157us) server_negotiation.cc:407] Connection header received
0807 18:37:13.721743 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:37:13.721747 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:37:13.721806 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:37:13.721850 (+ 44us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:37:13.722423 (+ 573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:13.723222 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:13.724565 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:13.725090 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:13.725241 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:13.725306 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0807 18:37:13.725453 (+ 147us) server_negotiation.cc:299] Negotiation successful
0807 18:37:13.725527 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260807 18:37:13.726378 30345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:13.721153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.66:37897 (local address 127.11.14.67:57283)
0807 18:37:13.721497 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0807 18:37:13.721512 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 18:37:13.721612 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:37:13.722000 (+ 388us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:37:13.722003 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:37:13.722044 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:37:13.722295 (+ 251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:37:13.722302 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:37:13.724297 (+ 1995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:37:13.724301 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:37:13.725118 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:37:13.725128 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:37:13.725966 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:37:13.725970 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:37:13.726087 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:13.726120 (+ 33us) client_negotiation.cc:769] Sending connection context
0807 18:37:13.726178 (+ 58us) client_negotiation.cc:240] Negotiation successful
0807 18:37:13.726259 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":290,"mutex_wait_us":58,"spinlock_wait_cycles":29696,"thread_start_us":149,"threads_started":1}
I20260807 18:37:13.726425 30360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:13.721619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.67:57283 (local address 127.11.14.66:37897)
0807 18:37:13.721747 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0807 18:37:13.721751 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:37:13.721765 (+ 14us) server_negotiation.cc:407] Connection header received
0807 18:37:13.721822 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:37:13.721825 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:37:13.721887 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:37:13.721974 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:37:13.723384 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:13.724172 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:13.725247 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:13.725847 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:13.725954 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:13.725991 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0807 18:37:13.726234 (+ 243us) server_negotiation.cc:299] Negotiation successful
0807 18:37:13.726316 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260807 18:37:13.726569 29991 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f82ae4ee88e4e5a9219e20422293658" candidate_uuid: "8e4f59ea9e2e459085a2add77c56362c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "144162b22242477cbc510beea0e162fc" is_pre_election: true
I20260807 18:37:13.726933 29991 raft_consensus.cc:2454] T 4f82ae4ee88e4e5a9219e20422293658 P 144162b22242477cbc510beea0e162fc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e4f59ea9e2e459085a2add77c56362c in term 0.
I20260807 18:37:13.727149 30144 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f82ae4ee88e4e5a9219e20422293658" candidate_uuid: "8e4f59ea9e2e459085a2add77c56362c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7eea4d71f76945ab9b338c8ab5b2442b" is_pre_election: true
I20260807 18:37:13.727448 30144 raft_consensus.cc:2454] T 4f82ae4ee88e4e5a9219e20422293658 P 7eea4d71f76945ab9b338c8ab5b2442b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e4f59ea9e2e459085a2add77c56362c in term 0.
I20260807 18:37:13.727468 30218 leader_election.cc:302] T 4f82ae4ee88e4e5a9219e20422293658 P 8e4f59ea9e2e459085a2add77c56362c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 144162b22242477cbc510beea0e162fc, 8e4f59ea9e2e459085a2add77c56362c; no voters:
I20260807 18:37:13.727756 30371 raft_consensus.cc:2781] T 4f82ae4ee88e4e5a9219e20422293658 P 8e4f59ea9e2e459085a2add77c56362c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:37:13.727854 30371 raft_consensus.cc:482] T 4f82ae4ee88e4e5a9219e20422293658 P 8e4f59ea9e2e459085a2add77c56362c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:37:13.727905 30371 raft_consensus.cc:3037] T 4f82ae4ee88e4e5a9219e20422293658 P 8e4f59ea9e2e459085a2add77c56362c [term 0 FOLLOWER]: Advancing to term 1
W20260807 18:37:13.728097 30344 tablet.cc:2369] T 4f82ae4ee88e4e5a9219e20422293658 P 8e4f59ea9e2e459085a2add77c56362c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:37:13.729049 30371 raft_consensus.cc:504] T 4f82ae4ee88e4e5a9219e20422293658 P 8e4f59ea9e2e459085a2add77c56362c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "144162b22242477cbc510beea0e162fc" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 40701 } } peers { permanent_uuid: "8e4f59ea9e2e459085a2add77c56362c" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 45015 } } peers { permanent_uuid: "7eea4d71f76945ab9b338c8ab5b2442b" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 37897 } }
I20260807 18:37:13.729435 30371 leader_election.cc:288] T 4f82ae4ee88e4e5a9219e20422293658 P 8e4f59ea9e2e459085a2add77c56362c [CANDIDATE]: Term 1 election: Requested vote from peers 144162b22242477cbc510beea0e162fc (127.11.14.65:40701), 7eea4d71f76945ab9b338c8ab5b2442b (127.11.14.66:37897)
I20260807 18:37:13.729785 29991 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f82ae4ee88e4e5a9219e20422293658" candidate_uuid: "8e4f59ea9e2e459085a2add77c56362c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "144162b22242477cbc510beea0e162fc"
I20260807 18:37:13.729835 30144 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f82ae4ee88e4e5a9219e20422293658" candidate_uuid: "8e4f59ea9e2e459085a2add77c56362c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7eea4d71f76945ab9b338c8ab5b2442b"
I20260807 18:37:13.729921 29991 raft_consensus.cc:3037] T 4f82ae4ee88e4e5a9219e20422293658 P 144162b22242477cbc510beea0e162fc [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:37:13.729993 30144 raft_consensus.cc:3037] T 4f82ae4ee88e4e5a9219e20422293658 P 7eea4d71f76945ab9b338c8ab5b2442b [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:37:13.731027 29991 raft_consensus.cc:2454] T 4f82ae4ee88e4e5a9219e20422293658 P 144162b22242477cbc510beea0e162fc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e4f59ea9e2e459085a2add77c56362c in term 1.
I20260807 18:37:13.731109 30144 raft_consensus.cc:2454] T 4f82ae4ee88e4e5a9219e20422293658 P 7eea4d71f76945ab9b338c8ab5b2442b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e4f59ea9e2e459085a2add77c56362c in term 1.
I20260807 18:37:13.731449 30222 leader_election.cc:302] T 4f82ae4ee88e4e5a9219e20422293658 P 8e4f59ea9e2e459085a2add77c56362c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7eea4d71f76945ab9b338c8ab5b2442b, 8e4f59ea9e2e459085a2add77c56362c; no voters:
I20260807 18:37:13.731670 30371 raft_consensus.cc:2781] T 4f82ae4ee88e4e5a9219e20422293658 P 8e4f59ea9e2e459085a2add77c56362c [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:37:13.731910 30371 raft_consensus.cc:686] T 4f82ae4ee88e4e5a9219e20422293658 P 8e4f59ea9e2e459085a2add77c56362c [term 1 LEADER]: Becoming Leader. State: Replica: 8e4f59ea9e2e459085a2add77c56362c, State: Running, Role: LEADER
I20260807 18:37:13.732299 30371 consensus_queue.cc:237] T 4f82ae4ee88e4e5a9219e20422293658 P 8e4f59ea9e2e459085a2add77c56362c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "144162b22242477cbc510beea0e162fc" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 40701 } } peers { permanent_uuid: "8e4f59ea9e2e459085a2add77c56362c" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 45015 } } peers { permanent_uuid: "7eea4d71f76945ab9b338c8ab5b2442b" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 37897 } }
I20260807 18:37:13.735662 29828 catalog_manager.cc:5495] T 4f82ae4ee88e4e5a9219e20422293658 P 8e4f59ea9e2e459085a2add77c56362c reported cstate change: term changed from 0 to 1, leader changed from <none> to 8e4f59ea9e2e459085a2add77c56362c (127.11.14.67). New cstate: current_term: 1 leader_uuid: "8e4f59ea9e2e459085a2add77c56362c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "144162b22242477cbc510beea0e162fc" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 40701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8e4f59ea9e2e459085a2add77c56362c" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 45015 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7eea4d71f76945ab9b338c8ab5b2442b" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 37897 } health_report { overall_health: UNKNOWN } } }
W20260807 18:37:13.746841 30195 tablet.cc:2369] T 4f82ae4ee88e4e5a9219e20422293658 P 7eea4d71f76945ab9b338c8ab5b2442b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:37:13.777483 30361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:13.773738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51836 (local address 127.11.14.67:45015)
0807 18:37:13.773887 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0807 18:37:13.773892 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:37:13.773906 (+ 14us) server_negotiation.cc:407] Connection header received
0807 18:37:13.774075 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:37:13.774077 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:37:13.774135 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:37:13.774248 (+ 113us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:37:13.774524 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:13.775032 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:13.775976 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:13.776191 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:13.776309 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:13.776641 (+ 332us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:37:13.777098 (+ 457us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:37:13.777233 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0807 18:37:13.777273 (+ 40us) server_negotiation.cc:299] Negotiation successful
0807 18:37:13.777332 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
W20260807 18:37:13.782697 30045 tablet.cc:2369] T 4f82ae4ee88e4e5a9219e20422293658 P 144162b22242477cbc510beea0e162fc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:37:13.783322 29991 raft_consensus.cc:1261] T 4f82ae4ee88e4e5a9219e20422293658 P 144162b22242477cbc510beea0e162fc [term 1 FOLLOWER]: Refusing update from remote peer 8e4f59ea9e2e459085a2add77c56362c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:37:13.783336 30144 raft_consensus.cc:1261] T 4f82ae4ee88e4e5a9219e20422293658 P 7eea4d71f76945ab9b338c8ab5b2442b [term 1 FOLLOWER]: Refusing update from remote peer 8e4f59ea9e2e459085a2add77c56362c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:37:13.784152 30371 consensus_queue.cc:1035] T 4f82ae4ee88e4e5a9219e20422293658 P 8e4f59ea9e2e459085a2add77c56362c [LEADER]: Connected to new peer: Peer: permanent_uuid: "144162b22242477cbc510beea0e162fc" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 40701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:37:13.784377 30374 consensus_queue.cc:1035] T 4f82ae4ee88e4e5a9219e20422293658 P 8e4f59ea9e2e459085a2add77c56362c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eea4d71f76945ab9b338c8ab5b2442b" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 37897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:37:13.800410 29830 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41988:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:37:13.800601 29830 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41988:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:37:13.803350 29830 catalog_manager.cc:5782] T 00000000000000000000000000000000 P eb70e53946e94041af95a06a4a8e6daf: Sending DeleteTablet for 3 replicas of tablet 4f82ae4ee88e4e5a9219e20422293658
I20260807 18:37:13.804097 30272 tablet_service.cc:1514] Processing DeleteTablet for tablet 4f82ae4ee88e4e5a9219e20422293658 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:37:13 UTC) from {username='kudu', principal='kudu/127.11.14.126@KRBTEST.COM'} at 127.0.0.1:51826
I20260807 18:37:13.804104 29969 tablet_service.cc:1514] Processing DeleteTablet for tablet 4f82ae4ee88e4e5a9219e20422293658 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:37:13 UTC) from {username='kudu', principal='kudu/127.11.14.126@KRBTEST.COM'} at 127.0.0.1:38012
I20260807 18:37:13.804436 30122 tablet_service.cc:1514] Processing DeleteTablet for tablet 4f82ae4ee88e4e5a9219e20422293658 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:37:13 UTC) from {username='kudu', principal='kudu/127.11.14.126@KRBTEST.COM'} at 127.0.0.1:50438
I20260807 18:37:13.804587 30390 tablet_replica.cc:330] T 4f82ae4ee88e4e5a9219e20422293658 P 144162b22242477cbc510beea0e162fc: stopping tablet replica
I20260807 18:37:13.804792 30389 tablet_replica.cc:330] T 4f82ae4ee88e4e5a9219e20422293658 P 8e4f59ea9e2e459085a2add77c56362c: stopping tablet replica
I20260807 18:37:13.804926 30390 raft_consensus.cc:2229] T 4f82ae4ee88e4e5a9219e20422293658 P 144162b22242477cbc510beea0e162fc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:37:13.805208 30389 raft_consensus.cc:2229] T 4f82ae4ee88e4e5a9219e20422293658 P 8e4f59ea9e2e459085a2add77c56362c [term 1 LEADER]: Raft consensus shutting down.
I20260807 18:37:13.805253 30390 raft_consensus.cc:2258] T 4f82ae4ee88e4e5a9219e20422293658 P 144162b22242477cbc510beea0e162fc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:37:13.805823 11321 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkz56XN/build/debug/bin/kudu with pid 29900
I20260807 18:37:13.812610 30391 tablet_replica.cc:330] T 4f82ae4ee88e4e5a9219e20422293658 P 7eea4d71f76945ab9b338c8ab5b2442b: stopping tablet replica
I20260807 18:37:13.812654 30389 raft_consensus.cc:2258] T 4f82ae4ee88e4e5a9219e20422293658 P 8e4f59ea9e2e459085a2add77c56362c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:37:13.812927 30391 raft_consensus.cc:2229] T 4f82ae4ee88e4e5a9219e20422293658 P 7eea4d71f76945ab9b338c8ab5b2442b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:37:13.813228 30391 raft_consensus.cc:2258] T 4f82ae4ee88e4e5a9219e20422293658 P 7eea4d71f76945ab9b338c8ab5b2442b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:37:13.814620 30389 ts_tablet_manager.cc:1905] T 4f82ae4ee88e4e5a9219e20422293658 P 8e4f59ea9e2e459085a2add77c56362c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:37:13.814891 30391 ts_tablet_manager.cc:1905] T 4f82ae4ee88e4e5a9219e20422293658 P 7eea4d71f76945ab9b338c8ab5b2442b: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260807 18:37:13.816391 29809 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.14.65:40701 (error 108)
I20260807 18:37:13.817307 11321 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkz56XN/build/debug/bin/kudu with pid 30050
I20260807 18:37:13.819484 30389 ts_tablet_manager.cc:1918] T 4f82ae4ee88e4e5a9219e20422293658 P 8e4f59ea9e2e459085a2add77c56362c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:37:13.819644 30389 log.cc:1196] T 4f82ae4ee88e4e5a9219e20422293658 P 8e4f59ea9e2e459085a2add77c56362c: Deleting WAL directory at /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786127579900523-11321-0/minicluster-data/ts-2/wal/wals/4f82ae4ee88e4e5a9219e20422293658
I20260807 18:37:13.820107 30389 ts_tablet_manager.cc:1939] T 4f82ae4ee88e4e5a9219e20422293658 P 8e4f59ea9e2e459085a2add77c56362c: Deleting consensus metadata
I20260807 18:37:13.821712 29813 catalog_manager.cc:4841] TS 8e4f59ea9e2e459085a2add77c56362c (127.11.14.67:45015): tablet 4f82ae4ee88e4e5a9219e20422293658 (table test-table [id=7707f9f41e4249808d8be6bbb7a5ee64]) successfully deleted
I20260807 18:37:13.825767 30356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:13.825435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.65:40701 (local address 127.0.0.1:38014)
0807 18:37:13.825573 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0807 18:37:13.825659 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.65:40701: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260807 18:37:13.826031 29809 catalog_manager.cc:4584] TS 144162b22242477cbc510beea0e162fc (127.11.14.65:40701): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 4f82ae4ee88e4e5a9219e20422293658: Network error: Client connection negotiation failed: client connection to 127.11.14.65:40701: connect: Connection refused (error 111)
I20260807 18:37:13.831144 11321 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkz56XN/build/debug/bin/kudu with pid 30199
I20260807 18:37:13.833601 30356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:13.831302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.66:37897 (local address 127.0.0.1:50450)
0807 18:37:13.833478 (+ 2176us) negotiation.cc:107] Waiting for socket to connect
0807 18:37:13.833528 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.66:37897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2097}
I20260807 18:37:13.839310 11321 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkz56XN/build/debug/bin/kudu with pid 29791
2026-08-07T18:37:13Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2639 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-07T18:37:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:37:13Z Disabled control of system clock
I20260807 18:37:13.875957 11321 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkz56XN/build/debug/bin/kudu
/tmp/dist-test-taskkz56XN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786127579900523-11321-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786127579900523-11321-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786127579900523-11321-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786127579900523-11321-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.14.126:39329
--webserver_interface=127.11.14.126
--webserver_port=0
--builtin_ntp_servers=127.11.14.84:37513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.14.126:39329
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 18:37:13.991063 30397 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:37:13.991314 30397 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:37:13.991350 30397 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:37:13.995343 30397 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:37:13.995401 30397 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:37:13.995417 30397 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:37:13.995466 30397 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:37:13.995498 30397 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:37:14.000267 30397 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.14.84:37513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786127579900523-11321-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786127579900523-11321-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.14.126:39329
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.14.126:39329
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786127579900523-11321-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.14.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30397
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786127579900523-11321-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:37:14.001379 30397 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:37:14.002880 30397 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:37:14.011529 30403 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:37:14.011607 30405 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:37:14.011839 30402 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:37:14.011650 30397 server_base.cc:1034] running on GCE node
I20260807 18:37:14.012414 30397 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:37:14.013535 30397 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:37:14.014743 30397 hybrid_clock.cc:648] HybridClock initialized: now 1786127834014716 us; error 45 us; skew 500 ppm
I20260807 18:37:14.016620 30397 webserver.cc:460] Webserver started at http://127.11.14.126:46601/ using document root <none> and password file <none>
I20260807 18:37:14.017215 30397 fs_manager.cc:359] Metadata directory not provided
I20260807 18:37:14.017305 30397 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:37:14.017529 30397 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:37:14.019649 30397 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786127579900523-11321-0/minicluster-data/master-0/data/instance:
uuid: "812d2d59f64645b68893671f6317fccb"
format_stamp: "Formatted at 2026-08-07 18:37:14 on dist-test-slave-6k22"
server_key: "6be35912a30e9927eafb96bc74a8d085"
server_key_iv: "d3e588ba796845c910408f3cee9ee395"
server_key_version: "encryptionkey@0"
I20260807 18:37:14.020248 30397 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786127579900523-11321-0/minicluster-data/master-0/wal/instance:
uuid: "812d2d59f64645b68893671f6317fccb"
format_stamp: "Formatted at 2026-08-07 18:37:14 on dist-test-slave-6k22"
server_key: "6be35912a30e9927eafb96bc74a8d085"
server_key_iv: "d3e588ba796845c910408f3cee9ee395"
server_key_version: "encryptionkey@0"
I20260807 18:37:14.023984 30397 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.003s
I20260807 18:37:14.026432 30413 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:37:14.027751 30397 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 18:37:14.027896 30397 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786127579900523-11321-0/minicluster-data/master-0/data,/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786127579900523-11321-0/minicluster-data/master-0/wal
uuid: "812d2d59f64645b68893671f6317fccb"
format_stamp: "Formatted at 2026-08-07 18:37:14 on dist-test-slave-6k22"
server_key: "6be35912a30e9927eafb96bc74a8d085"
server_key_iv: "d3e588ba796845c910408f3cee9ee395"
server_key_version: "encryptionkey@0"
I20260807 18:37:14.028005 30397 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786127579900523-11321-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786127579900523-11321-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786127579900523-11321-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:37:14.044852 30397 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:37:14.045560 30397 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:37:14.045759 30397 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:37:14.053297 30397 rpc_server.cc:307] RPC server started. Bound to: 127.11.14.126:39329
I20260807 18:37:14.053314 30469 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.14.126:39329 every 8 connection(s)
I20260807 18:37:14.054322 30397 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786127579900523-11321-0/minicluster-data/master-0/data/info.pb
I20260807 18:37:14.057461 30470 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:37:14.062001 11321 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkz56XN/build/debug/bin/kudu as pid 30397
I20260807 18:37:14.062173 11321 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786127579900523-11321-0/minicluster-data/master-0/wal/instance
I20260807 18:37:14.062438 11321 external_mini_cluster.cc:1428] Setting key 41c973388924b30dc0d1bc965e82faaf
I20260807 18:37:14.063423 30470 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 812d2d59f64645b68893671f6317fccb: Bootstrap starting.
I20260807 18:37:14.066172 30470 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 812d2d59f64645b68893671f6317fccb: Neither blocks nor log segments found. Creating new log.
I20260807 18:37:14.067095 30470 log.cc:824] T 00000000000000000000000000000000 P 812d2d59f64645b68893671f6317fccb: Log is configured to *not* fsync() on all Append() calls
I20260807 18:37:14.069206 30470 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 812d2d59f64645b68893671f6317fccb: No bootstrap required, opened a new log
I20260807 18:37:14.073549 30474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:14.063983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54472 (local address 127.11.14.126:39329)
0807 18:37:14.064718 (+ 735us) server_negotiation.cc:206] Beginning negotiation
0807 18:37:14.064741 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0807 18:37:14.064786 (+ 45us) server_negotiation.cc:407] Connection header received
0807 18:37:14.065630 (+ 844us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:37:14.065654 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:37:14.066161 (+ 507us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:37:14.066562 (+ 401us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:37:14.067025 (+ 463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:14.068440 (+ 1415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:14.069528 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:14.070205 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:14.070382 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:14.071253 (+ 871us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:37:14.071260 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:37:14.071285 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:37:14.071808 (+ 523us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:37:14.071881 (+ 73us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:37:14.071909 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:37:14.072010 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0807 18:37:14.072165 (+ 155us) server_negotiation.cc:299] Negotiation successful
0807 18:37:14.072486 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":277,"thread_start_us":110,"threads_started":1}
I20260807 18:37:14.073709 30470 raft_consensus.cc:348] T 00000000000000000000000000000000 P 812d2d59f64645b68893671f6317fccb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "812d2d59f64645b68893671f6317fccb" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 39329 } }
I20260807 18:37:14.073993 30470 raft_consensus.cc:374] T 00000000000000000000000000000000 P 812d2d59f64645b68893671f6317fccb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:37:14.074071 30470 raft_consensus.cc:729] T 00000000000000000000000000000000 P 812d2d59f64645b68893671f6317fccb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 812d2d59f64645b68893671f6317fccb, State: Initialized, Role: FOLLOWER
I20260807 18:37:14.074479 30470 consensus_queue.cc:260] T 00000000000000000000000000000000 P 812d2d59f64645b68893671f6317fccb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "812d2d59f64645b68893671f6317fccb" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 39329 } }
I20260807 18:37:14.074628 30470 raft_consensus.cc:388] T 00000000000000000000000000000000 P 812d2d59f64645b68893671f6317fccb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:37:14.074707 30470 raft_consensus.cc:482] T 00000000000000000000000000000000 P 812d2d59f64645b68893671f6317fccb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:37:14.074836 30470 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 812d2d59f64645b68893671f6317fccb [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:37:14.075807 30470 raft_consensus.cc:504] T 00000000000000000000000000000000 P 812d2d59f64645b68893671f6317fccb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "812d2d59f64645b68893671f6317fccb" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 39329 } }
I20260807 18:37:14.076141 30470 leader_election.cc:302] T 00000000000000000000000000000000 P 812d2d59f64645b68893671f6317fccb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 812d2d59f64645b68893671f6317fccb; no voters:
I20260807 18:37:14.076396 30470 leader_election.cc:288] T 00000000000000000000000000000000 P 812d2d59f64645b68893671f6317fccb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:37:14.076505 30476 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 812d2d59f64645b68893671f6317fccb [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:37:14.076740 30476 raft_consensus.cc:686] T 00000000000000000000000000000000 P 812d2d59f64645b68893671f6317fccb [term 1 LEADER]: Becoming Leader. State: Replica: 812d2d59f64645b68893671f6317fccb, State: Running, Role: LEADER
I20260807 18:37:14.077071 30476 consensus_queue.cc:237] T 00000000000000000000000000000000 P 812d2d59f64645b68893671f6317fccb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "812d2d59f64645b68893671f6317fccb" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 39329 } }
I20260807 18:37:14.077613 30470 sys_catalog.cc:564] T 00000000000000000000000000000000 P 812d2d59f64645b68893671f6317fccb [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:37:14.078670 30477 sys_catalog.cc:455] T 00000000000000000000000000000000 P 812d2d59f64645b68893671f6317fccb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "812d2d59f64645b68893671f6317fccb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "812d2d59f64645b68893671f6317fccb" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 39329 } } }
I20260807 18:37:14.078814 30477 sys_catalog.cc:458] T 00000000000000000000000000000000 P 812d2d59f64645b68893671f6317fccb [sys.catalog]: This master's current role is: LEADER
I20260807 18:37:14.079128 30478 sys_catalog.cc:455] T 00000000000000000000000000000000 P 812d2d59f64645b68893671f6317fccb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 812d2d59f64645b68893671f6317fccb. Latest consensus state: current_term: 1 leader_uuid: "812d2d59f64645b68893671f6317fccb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "812d2d59f64645b68893671f6317fccb" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 39329 } } }
I20260807 18:37:14.079232 30485 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:37:14.079233 30478 sys_catalog.cc:458] T 00000000000000000000000000000000 P 812d2d59f64645b68893671f6317fccb [sys.catalog]: This master's current role is: LEADER
I20260807 18:37:14.082382 30485 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:37:14.087986 30485 catalog_manager.cc:1347] Generated new cluster ID: f5e32330dd474bc5b9261b97b83b196c
I20260807 18:37:14.088073 30485 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:37:14.099167 30485 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:37:14.100395 30485 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:37:14.113672 30485 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 812d2d59f64645b68893671f6317fccb: Generated new TSK 0
I20260807 18:37:14.114394 30485 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:37:14.120290 11321 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkz56XN/build/debug/bin/kudu
/tmp/dist-test-taskkz56XN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786127579900523-11321-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786127579900523-11321-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786127579900523-11321-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786127579900523-11321-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.14.65:0
--local_ip_for_outbound_sockets=127.11.14.65
--webserver_interface=127.11.14.65
--webserver_port=0
--tserver_master_addrs=127.11.14.126:39329
--builtin_ntp_servers=127.11.14.84:37513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 18:37:14.283782 30497 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:37:14.284071 30497 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:37:14.284142 30497 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:37:14.288285 30497 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:37:14.288336 30497 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:37:14.288430 30497 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.14.65
I20260807 18:37:14.293195 30497 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.14.84:37513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786127579900523-11321-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786127579900523-11321-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.14.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786127579900523-11321-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.14.65
--webserver_port=0
--tserver_master_addrs=127.11.14.126:39329
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30497
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.14.65
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786127579900523-11321-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:37:14.294278 30497 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:37:14.295706 30497 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:37:14.305220 30504 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:37:14.305266 30506 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:37:14.305220 30503 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:37:14.305290 30497 server_base.cc:1034] running on GCE node
I20260807 18:37:14.305873 30497 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:37:14.306586 30497 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:37:14.307760 30497 hybrid_clock.cc:648] HybridClock initialized: now 1786127834307735 us; error 31 us; skew 500 ppm
I20260807 18:37:14.309553 30497 webserver.cc:460] Webserver started at http://127.11.14.65:42433/ using document root <none> and password file <none>
I20260807 18:37:14.310130 30497 fs_manager.cc:359] Metadata directory not provided
I20260807 18:37:14.310192 30497 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:37:14.310418 30497 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:37:14.312505 30497 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786127579900523-11321-0/minicluster-data/ts-0/data/instance:
uuid: "498232f6bbe549a7a8284e3c51f05d02"
format_stamp: "Formatted at 2026-08-07 18:37:14 on dist-test-slave-6k22"
server_key: "a8fc3e639fbe3b751711047012907bee"
server_key_iv: "db6be43f22a54037ccf5a6ec7da77e77"
server_key_version: "encryptionkey@0"
I20260807 18:37:14.313117 30497 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786127579900523-11321-0/minicluster-data/ts-0/wal/instance:
uuid: "498232f6bbe549a7a8284e3c51f05d02"
format_stamp: "Formatted at 2026-08-07 18:37:14 on dist-test-slave-6k22"
server_key: "a8fc3e639fbe3b751711047012907bee"
server_key_iv: "db6be43f22a54037ccf5a6ec7da77e77"
server_key_version: "encryptionkey@0"
I20260807 18:37:14.316820 30497 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 18:37:14.319258 30514 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:37:14.320569 30497 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 18:37:14.320670 30497 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786127579900523-11321-0/minicluster-data/ts-0/data,/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786127579900523-11321-0/minicluster-data/ts-0/wal
uuid: "498232f6bbe549a7a8284e3c51f05d02"
format_stamp: "Formatted at 2026-08-07 18:37:14 on dist-test-slave-6k22"
server_key: "a8fc3e639fbe3b751711047012907bee"
server_key_iv: "db6be43f22a54037ccf5a6ec7da77e77"
server_key_version: "encryptionkey@0"
I20260807 18:37:14.320806 30497 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786127579900523-11321-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786127579900523-11321-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786127579900523-11321-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:37:14.334633 30497 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:37:14.335294 30497 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:37:14.335485 30497 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:37:14.336088 30497 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:37:14.337016 30497 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:37:14.337065 30497 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:37:14.337129 30497 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:37:14.337162 30497 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:37:14.345755 30497 rpc_server.cc:307] RPC server started. Bound to: 127.11.14.65:39787
I20260807 18:37:14.345845 30644 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.14.65:39787 every 8 connection(s)
I20260807 18:37:14.346696 30497 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786127579900523-11321-0/minicluster-data/ts-0/data/info.pb
I20260807 18:37:14.348618 11321 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkz56XN/build/debug/bin/kudu as pid 30497
I20260807 18:37:14.348747 11321 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786127579900523-11321-0/minicluster-data/ts-0/wal/instance
I20260807 18:37:14.349041 11321 external_mini_cluster.cc:1428] Setting key 82d61449b594115f3d3b2e5a38ba51c4
I20260807 18:37:14.351281 11321 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkz56XN/build/debug/bin/kudu
/tmp/dist-test-taskkz56XN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786127579900523-11321-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786127579900523-11321-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786127579900523-11321-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786127579900523-11321-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.14.66:0
--local_ip_for_outbound_sockets=127.11.14.66
--webserver_interface=127.11.14.66
--webserver_port=0
--tserver_master_addrs=127.11.14.126:39329
--builtin_ntp_servers=127.11.14.84:37513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 18:37:14.357584 30474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:14.348634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.65:41875 (local address 127.11.14.126:39329)
0807 18:37:14.348816 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0807 18:37:14.348823 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:37:14.350006 (+ 1183us) server_negotiation.cc:407] Connection header received
0807 18:37:14.351936 (+ 1930us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:37:14.351938 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:37:14.351980 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:37:14.352050 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:37:14.353265 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:14.353810 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:14.355404 (+ 1594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:14.355681 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:14.355765 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:14.357118 (+ 1353us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:37:14.357121 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:37:14.357123 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:37:14.357135 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:37:14.357153 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:37:14.357161 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:37:14.357221 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0807 18:37:14.357421 (+ 200us) server_negotiation.cc:299] Negotiation successful
0807 18:37:14.357473 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260807 18:37:14.358945 30647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:14.349047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:39329 (local address 127.11.14.65:41875)
0807 18:37:14.349858 (+ 811us) negotiation.cc:107] Waiting for socket to connect
0807 18:37:14.349912 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0807 18:37:14.351695 (+ 1783us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:37:14.352224 (+ 529us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:37:14.352235 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:37:14.352347 (+ 112us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:37:14.353066 (+ 719us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:37:14.353091 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:37:14.353938 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:37:14.353942 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:37:14.355274 (+ 1332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:37:14.355289 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:37:14.355846 (+ 557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:37:14.355854 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:37:14.356130 (+ 276us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:14.356791 (+ 661us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 18:37:14.356794 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:37:14.356857 (+ 63us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 18:37:14.356951 (+ 94us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:37:14.357248 (+ 297us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:37:14.357261 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:37:14.357271 (+ 10us) client_negotiation.cc:769] Sending connection context
0807 18:37:14.357418 (+ 147us) client_negotiation.cc:240] Negotiation successful
0807 18:37:14.357753 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":332,"thread_start_us":146,"threads_started":1}
I20260807 18:37:14.360251 30645 heartbeater.cc:347] Connected to a master server at 127.11.14.126:39329
I20260807 18:37:14.360500 30645 heartbeater.cc:464] Registering TS with master...
I20260807 18:37:14.361037 30645 heartbeater.cc:511] Master 127.11.14.126:39329 requested a full tablet report, sending...
I20260807 18:37:14.362524 30431 ts_manager.cc:194] Registered new tserver with Master: 498232f6bbe549a7a8284e3c51f05d02 (127.11.14.65:39787)
I20260807 18:37:14.363777 30431 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.14.65:41875
W20260807 18:37:14.480976 30648 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:37:14.481307 30648 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:37:14.481397 30648 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:37:14.485562 30648 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:37:14.485661 30648 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:37:14.485762 30648 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.14.66
I20260807 18:37:14.490761 30648 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.14.84:37513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786127579900523-11321-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786127579900523-11321-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.14.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786127579900523-11321-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.14.66
--webserver_port=0
--tserver_master_addrs=127.11.14.126:39329
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30648
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.14.66
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786127579900523-11321-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:37:14.491999 30648 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:37:14.493446 30648 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:37:14.503068 30653 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:37:14.503062 30656 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:37:14.503217 30654 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:37:14.503542 30648 server_base.cc:1034] running on GCE node
I20260807 18:37:14.503979 30648 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:37:14.504606 30648 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:37:14.505801 30648 hybrid_clock.cc:648] HybridClock initialized: now 1786127834505782 us; error 34 us; skew 500 ppm
I20260807 18:37:14.507726 30648 webserver.cc:460] Webserver started at http://127.11.14.66:44607/ using document root <none> and password file <none>
I20260807 18:37:14.508313 30648 fs_manager.cc:359] Metadata directory not provided
I20260807 18:37:14.508407 30648 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:37:14.508636 30648 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:37:14.510706 30648 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786127579900523-11321-0/minicluster-data/ts-1/data/instance:
uuid: "fdff3b07c7cf4746a694ab2ddff2ec63"
format_stamp: "Formatted at 2026-08-07 18:37:14 on dist-test-slave-6k22"
server_key: "e149dfb1b7f646f65c54462bc68789eb"
server_key_iv: "babe0e0ace5d23d20ad4768fcb9e8043"
server_key_version: "encryptionkey@0"
I20260807 18:37:14.511302 30648 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786127579900523-11321-0/minicluster-data/ts-1/wal/instance:
uuid: "fdff3b07c7cf4746a694ab2ddff2ec63"
format_stamp: "Formatted at 2026-08-07 18:37:14 on dist-test-slave-6k22"
server_key: "e149dfb1b7f646f65c54462bc68789eb"
server_key_iv: "babe0e0ace5d23d20ad4768fcb9e8043"
server_key_version: "encryptionkey@0"
I20260807 18:37:14.515115 30648 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260807 18:37:14.517611 30666 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:37:14.518904 30648 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 18:37:14.519047 30648 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786127579900523-11321-0/minicluster-data/ts-1/data,/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786127579900523-11321-0/minicluster-data/ts-1/wal
uuid: "fdff3b07c7cf4746a694ab2ddff2ec63"
format_stamp: "Formatted at 2026-08-07 18:37:14 on dist-test-slave-6k22"
server_key: "e149dfb1b7f646f65c54462bc68789eb"
server_key_iv: "babe0e0ace5d23d20ad4768fcb9e8043"
server_key_version: "encryptionkey@0"
I20260807 18:37:14.519162 30648 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786127579900523-11321-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786127579900523-11321-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786127579900523-11321-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:37:14.543955 30648 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:37:14.544700 30648 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:37:14.544921 30648 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:37:14.545548 30648 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:37:14.546535 30648 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:37:14.546607 30648 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:37:14.546679 30648 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:37:14.546722 30648 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:37:14.555518 30648 rpc_server.cc:307] RPC server started. Bound to: 127.11.14.66:46339
I20260807 18:37:14.555620 30791 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.14.66:46339 every 8 connection(s)
I20260807 18:37:14.556485 30648 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786127579900523-11321-0/minicluster-data/ts-1/data/info.pb
I20260807 18:37:14.559748 11321 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkz56XN/build/debug/bin/kudu as pid 30648
I20260807 18:37:14.559877 11321 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786127579900523-11321-0/minicluster-data/ts-1/wal/instance
I20260807 18:37:14.560163 11321 external_mini_cluster.cc:1428] Setting key cb63f59b9ddc6cdc767e6c01ecada3c1
I20260807 18:37:14.562335 11321 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkz56XN/build/debug/bin/kudu
/tmp/dist-test-taskkz56XN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786127579900523-11321-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786127579900523-11321-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786127579900523-11321-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786127579900523-11321-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.14.67:0
--local_ip_for_outbound_sockets=127.11.14.67
--webserver_interface=127.11.14.67
--webserver_port=0
--tserver_master_addrs=127.11.14.126:39329
--builtin_ntp_servers=127.11.14.84:37513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 18:37:14.566972 30474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:14.558368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.66:44639 (local address 127.11.14.126:39329)
0807 18:37:14.558498 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0807 18:37:14.558507 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:37:14.559593 (+ 1086us) server_negotiation.cc:407] Connection header received
0807 18:37:14.561322 (+ 1729us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:37:14.561325 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:37:14.561366 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:37:14.561442 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:37:14.562550 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:14.563071 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:14.564642 (+ 1571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:14.564960 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:14.565053 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:14.566386 (+ 1333us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:37:14.566393 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:37:14.566399 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:37:14.566417 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:37:14.566445 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:37:14.566458 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:37:14.566541 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0807 18:37:14.566689 (+ 148us) server_negotiation.cc:299] Negotiation successful
0807 18:37:14.566762 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260807 18:37:14.567838 30794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:14.558728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:39329 (local address 127.11.14.66:44639)
0807 18:37:14.559417 (+ 689us) negotiation.cc:107] Waiting for socket to connect
0807 18:37:14.559456 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0807 18:37:14.561088 (+ 1632us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:37:14.561723 (+ 635us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:37:14.561733 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:37:14.561846 (+ 113us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:37:14.562357 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:37:14.562369 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:37:14.563202 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:37:14.563205 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:37:14.564503 (+ 1298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:37:14.564519 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:37:14.565135 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:37:14.565142 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:37:14.565386 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:14.566103 (+ 717us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 18:37:14.566105 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:37:14.566147 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 18:37:14.566212 (+ 65us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:37:14.566548 (+ 336us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:37:14.566556 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:37:14.566561 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 18:37:14.566661 (+ 100us) client_negotiation.cc:240] Negotiation successful
0807 18:37:14.566891 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":286,"thread_start_us":127,"threads_started":1}
I20260807 18:37:14.569211 30792 heartbeater.cc:347] Connected to a master server at 127.11.14.126:39329
I20260807 18:37:14.569489 30792 heartbeater.cc:464] Registering TS with master...
I20260807 18:37:14.570076 30792 heartbeater.cc:511] Master 127.11.14.126:39329 requested a full tablet report, sending...
I20260807 18:37:14.571228 30431 ts_manager.cc:194] Registered new tserver with Master: fdff3b07c7cf4746a694ab2ddff2ec63 (127.11.14.66:46339)
I20260807 18:37:14.571940 30431 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.14.66:44639
W20260807 18:37:14.682842 30796 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:37:14.683107 30796 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:37:14.683146 30796 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:37:14.687129 30796 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:37:14.687188 30796 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:37:14.687242 30796 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.14.67
I20260807 18:37:14.692091 30796 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.14.84:37513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786127579900523-11321-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786127579900523-11321-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.14.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786127579900523-11321-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.14.67
--webserver_port=0
--tserver_master_addrs=127.11.14.126:39329
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30796
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.14.67
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786127579900523-11321-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:37:14.693235 30796 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:37:14.694692 30796 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:37:14.704903 30804 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:37:14.704913 30802 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:37:14.704921 30801 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:37:14.705214 30796 server_base.cc:1034] running on GCE node
I20260807 18:37:14.705662 30796 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:37:14.706266 30796 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:37:14.707461 30796 hybrid_clock.cc:648] HybridClock initialized: now 1786127834707420 us; error 52 us; skew 500 ppm
I20260807 18:37:14.709414 30796 webserver.cc:460] Webserver started at http://127.11.14.67:38415/ using document root <none> and password file <none>
I20260807 18:37:14.710065 30796 fs_manager.cc:359] Metadata directory not provided
I20260807 18:37:14.710134 30796 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:37:14.710386 30796 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:37:14.712594 30796 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786127579900523-11321-0/minicluster-data/ts-2/data/instance:
uuid: "8a926d64ca6f4488bdeef5143e6675d7"
format_stamp: "Formatted at 2026-08-07 18:37:14 on dist-test-slave-6k22"
server_key: "fd487e60d5c2babde6579c9966a7ac68"
server_key_iv: "98457073240576f301fcb211971d3020"
server_key_version: "encryptionkey@0"
I20260807 18:37:14.713199 30796 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786127579900523-11321-0/minicluster-data/ts-2/wal/instance:
uuid: "8a926d64ca6f4488bdeef5143e6675d7"
format_stamp: "Formatted at 2026-08-07 18:37:14 on dist-test-slave-6k22"
server_key: "fd487e60d5c2babde6579c9966a7ac68"
server_key_iv: "98457073240576f301fcb211971d3020"
server_key_version: "encryptionkey@0"
I20260807 18:37:14.717051 30796 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 18:37:14.719467 30812 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:37:14.721222 30796 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:37:14.721347 30796 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786127579900523-11321-0/minicluster-data/ts-2/data,/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786127579900523-11321-0/minicluster-data/ts-2/wal
uuid: "8a926d64ca6f4488bdeef5143e6675d7"
format_stamp: "Formatted at 2026-08-07 18:37:14 on dist-test-slave-6k22"
server_key: "fd487e60d5c2babde6579c9966a7ac68"
server_key_iv: "98457073240576f301fcb211971d3020"
server_key_version: "encryptionkey@0"
I20260807 18:37:14.721462 30796 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786127579900523-11321-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786127579900523-11321-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786127579900523-11321-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:37:14.738407 30796 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:37:14.739151 30796 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:37:14.739364 30796 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:37:14.740252 30796 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:37:14.741304 30796 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:37:14.741357 30796 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:37:14.741429 30796 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:37:14.741463 30796 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:37:14.750274 30796 rpc_server.cc:307] RPC server started. Bound to: 127.11.14.67:37977
I20260807 18:37:14.750380 30935 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.14.67:37977 every 8 connection(s)
I20260807 18:37:14.751317 30796 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786127579900523-11321-0/minicluster-data/ts-2/data/info.pb
I20260807 18:37:14.760983 11321 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkz56XN/build/debug/bin/kudu as pid 30796
I20260807 18:37:14.761138 11321 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786127579900523-11321-0/minicluster-data/ts-2/wal/instance
I20260807 18:37:14.761281 30474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:14.753402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.67:43061 (local address 127.11.14.126:39329)
0807 18:37:14.753587 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0807 18:37:14.753592 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:37:14.754746 (+ 1154us) server_negotiation.cc:407] Connection header received
0807 18:37:14.756293 (+ 1547us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:37:14.756296 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:37:14.756347 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:37:14.756427 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:37:14.757430 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:14.757960 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:14.759158 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:14.759371 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:14.759456 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:14.760730 (+ 1274us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:37:14.760734 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:37:14.760737 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:37:14.760753 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:37:14.760804 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:37:14.760816 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:37:14.760895 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0807 18:37:14.761051 (+ 156us) server_negotiation.cc:299] Negotiation successful
0807 18:37:14.761121 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260807 18:37:14.761487 11321 external_mini_cluster.cc:1428] Setting key d762544affe89097cc7db6b34c8d8642
I20260807 18:37:14.762182 30939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:14.753832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:39329 (local address 127.11.14.67:43061)
0807 18:37:14.754567 (+ 735us) negotiation.cc:107] Waiting for socket to connect
0807 18:37:14.754619 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0807 18:37:14.756085 (+ 1466us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:37:14.756609 (+ 524us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:37:14.756617 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:37:14.756772 (+ 155us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:37:14.757297 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:37:14.757310 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:37:14.758086 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:37:14.758089 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:37:14.759046 (+ 957us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:37:14.759056 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:37:14.759544 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:37:14.759547 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:37:14.759772 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:14.760455 (+ 683us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 18:37:14.760457 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:37:14.760496 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 18:37:14.760563 (+ 67us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:37:14.760901 (+ 338us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:37:14.760909 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:37:14.760913 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 18:37:14.761011 (+ 98us) client_negotiation.cc:240] Negotiation successful
0807 18:37:14.761310 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":301,"thread_start_us":116,"threads_started":1}
I20260807 18:37:14.763406 30936 heartbeater.cc:347] Connected to a master server at 127.11.14.126:39329
I20260807 18:37:14.763676 30936 heartbeater.cc:464] Registering TS with master...
I20260807 18:37:14.764214 30936 heartbeater.cc:511] Master 127.11.14.126:39329 requested a full tablet report, sending...
I20260807 18:37:14.765295 30431 ts_manager.cc:194] Registered new tserver with Master: 8a926d64ca6f4488bdeef5143e6675d7 (127.11.14.67:37977)
I20260807 18:37:14.765825 30431 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.14.67:43061
I20260807 18:37:14.775398 11321 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 18:37:14.781636 30474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:14.777433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54478 (local address 127.11.14.126:39329)
0807 18:37:14.777582 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0807 18:37:14.777587 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:37:14.777742 (+ 155us) server_negotiation.cc:407] Connection header received
0807 18:37:14.777912 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:37:14.777915 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:37:14.777965 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:37:14.778047 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:37:14.778379 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:14.779011 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:14.779992 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:14.780231 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:14.780348 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:14.781129 (+ 781us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:37:14.781132 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:37:14.781134 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:37:14.781166 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:37:14.781184 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:37:14.781192 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:37:14.781270 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0807 18:37:14.781425 (+ 155us) server_negotiation.cc:299] Negotiation successful
0807 18:37:14.781482 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260807 18:37:14.784596 30431 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:54478:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 18:37:14.787621 30431 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 18:37:14.801833 30955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:14.796669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50710 (local address 127.11.14.66:46339)
0807 18:37:14.797054 (+ 385us) server_negotiation.cc:206] Beginning negotiation
0807 18:37:14.797074 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0807 18:37:14.797094 (+ 20us) server_negotiation.cc:407] Connection header received
0807 18:37:14.797188 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:37:14.797194 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:37:14.797400 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:37:14.797876 (+ 476us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:37:14.798006 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:14.799065 (+ 1059us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:14.800587 (+ 1522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:14.801126 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:14.801257 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:14.801481 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0807 18:37:14.801548 (+ 67us) server_negotiation.cc:299] Negotiation successful
0807 18:37:14.801657 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":279,"thread_start_us":141,"threads_started":1}
I20260807 18:37:14.801828 30949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:14.795889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.66:46339 (local address 127.0.0.1:50710)
0807 18:37:14.796707 (+ 818us) negotiation.cc:107] Waiting for socket to connect
0807 18:37:14.796720 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 18:37:14.796793 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:37:14.797565 (+ 772us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:37:14.797571 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:37:14.797592 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:37:14.797720 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:37:14.797727 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:37:14.799187 (+ 1460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:37:14.799190 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:37:14.800447 (+ 1257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:37:14.800458 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:37:14.801258 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:37:14.801262 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:37:14.801371 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:14.801408 (+ 37us) client_negotiation.cc:769] Sending connection context
0807 18:37:14.801509 (+ 101us) client_negotiation.cc:240] Negotiation successful
0807 18:37:14.801665 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":759,"mutex_wait_us":79,"thread_start_us":103,"threads_started":1}
I20260807 18:37:14.801964 30951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:14.795815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.67:37977 (local address 127.0.0.1:53614)
0807 18:37:14.796277 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0807 18:37:14.796352 (+ 75us) client_negotiation.cc:174] Beginning negotiation
0807 18:37:14.796492 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:37:14.797189 (+ 697us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:37:14.797195 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:37:14.797219 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:37:14.797369 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:37:14.797377 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:37:14.798894 (+ 1517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:37:14.798897 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:37:14.800377 (+ 1480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:37:14.800390 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:37:14.801471 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:37:14.801475 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:37:14.801572 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:14.801629 (+ 57us) client_negotiation.cc:769] Sending connection context
0807 18:37:14.801679 (+ 50us) client_negotiation.cc:240] Negotiation successful
0807 18:37:14.801761 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":283,"mutex_wait_us":90,"thread_start_us":71,"threads_started":1}
I20260807 18:37:14.802104 30953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:14.796571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.65:39787 (local address 127.0.0.1:40530)
0807 18:37:14.797521 (+ 950us) negotiation.cc:107] Waiting for socket to connect
0807 18:37:14.797536 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 18:37:14.797623 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:37:14.798386 (+ 763us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:37:14.798389 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:37:14.798409 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:37:14.798493 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:37:14.798499 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:37:14.799604 (+ 1105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:37:14.799608 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:37:14.800734 (+ 1126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:37:14.800743 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:37:14.801749 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:37:14.801753 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:37:14.801852 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:14.801879 (+ 27us) client_negotiation.cc:769] Sending connection context
0807 18:37:14.801926 (+ 47us) client_negotiation.cc:240] Negotiation successful
0807 18:37:14.801992 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":892,"mutex_wait_us":22,"thread_start_us":87,"threads_started":1}
I20260807 18:37:14.802639 30952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:14.795938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53614 (local address 127.11.14.67:37977)
0807 18:37:14.796630 (+ 692us) server_negotiation.cc:206] Beginning negotiation
0807 18:37:14.796644 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 18:37:14.796668 (+ 24us) server_negotiation.cc:407] Connection header received
0807 18:37:14.796749 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:37:14.796755 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:37:14.797020 (+ 265us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:37:14.797197 (+ 177us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:37:14.797523 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:14.798752 (+ 1229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:14.800523 (+ 1771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:14.801327 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:14.802099 (+ 772us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:14.802332 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0807 18:37:14.802387 (+ 55us) server_negotiation.cc:299] Negotiation successful
0807 18:37:14.802481 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":575,"thread_start_us":98,"threads_started":1}
I20260807 18:37:14.802806 30956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:14.797095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40530 (local address 127.11.14.65:39787)
0807 18:37:14.797890 (+ 795us) server_negotiation.cc:206] Beginning negotiation
0807 18:37:14.797902 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 18:37:14.797920 (+ 18us) server_negotiation.cc:407] Connection header received
0807 18:37:14.797980 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:37:14.797984 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:37:14.798168 (+ 184us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:37:14.798406 (+ 238us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:37:14.798618 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:14.799468 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:14.800878 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:14.801620 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:14.802326 (+ 706us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:14.802494 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0807 18:37:14.802538 (+ 44us) server_negotiation.cc:299] Negotiation successful
0807 18:37:14.802621 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":692,"thread_start_us":163,"threads_started":1}
I20260807 18:37:14.805259 30864 tablet_service.cc:1467] Processing CreateTablet for tablet 2125fe6835ea47478b7206b78a54c11a (DEFAULT_TABLE table=test-table [id=73b5edcbf9f14cf5851d0576f1def04c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:37:14.805430 30722 tablet_service.cc:1467] Processing CreateTablet for tablet 2125fe6835ea47478b7206b78a54c11a (DEFAULT_TABLE table=test-table [id=73b5edcbf9f14cf5851d0576f1def04c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:37:14.805430 30571 tablet_service.cc:1467] Processing CreateTablet for tablet 2125fe6835ea47478b7206b78a54c11a (DEFAULT_TABLE table=test-table [id=73b5edcbf9f14cf5851d0576f1def04c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:37:14.806320 30864 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2125fe6835ea47478b7206b78a54c11a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:37:14.806460 30571 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2125fe6835ea47478b7206b78a54c11a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:37:14.806460 30722 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2125fe6835ea47478b7206b78a54c11a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:37:14.812923 30957 tablet_bootstrap.cc:492] T 2125fe6835ea47478b7206b78a54c11a P 8a926d64ca6f4488bdeef5143e6675d7: Bootstrap starting.
I20260807 18:37:14.814438 30959 tablet_bootstrap.cc:492] T 2125fe6835ea47478b7206b78a54c11a P fdff3b07c7cf4746a694ab2ddff2ec63: Bootstrap starting.
I20260807 18:37:14.814638 30961 tablet_bootstrap.cc:492] T 2125fe6835ea47478b7206b78a54c11a P 498232f6bbe549a7a8284e3c51f05d02: Bootstrap starting.
I20260807 18:37:14.815285 30957 tablet_bootstrap.cc:654] T 2125fe6835ea47478b7206b78a54c11a P 8a926d64ca6f4488bdeef5143e6675d7: Neither blocks nor log segments found. Creating new log.
I20260807 18:37:14.816138 30957 log.cc:824] T 2125fe6835ea47478b7206b78a54c11a P 8a926d64ca6f4488bdeef5143e6675d7: Log is configured to *not* fsync() on all Append() calls
I20260807 18:37:14.816753 30961 tablet_bootstrap.cc:654] T 2125fe6835ea47478b7206b78a54c11a P 498232f6bbe549a7a8284e3c51f05d02: Neither blocks nor log segments found. Creating new log.
I20260807 18:37:14.816753 30959 tablet_bootstrap.cc:654] T 2125fe6835ea47478b7206b78a54c11a P fdff3b07c7cf4746a694ab2ddff2ec63: Neither blocks nor log segments found. Creating new log.
I20260807 18:37:14.817631 30961 log.cc:824] T 2125fe6835ea47478b7206b78a54c11a P 498232f6bbe549a7a8284e3c51f05d02: Log is configured to *not* fsync() on all Append() calls
I20260807 18:37:14.817667 30959 log.cc:824] T 2125fe6835ea47478b7206b78a54c11a P fdff3b07c7cf4746a694ab2ddff2ec63: Log is configured to *not* fsync() on all Append() calls
I20260807 18:37:14.818270 30957 tablet_bootstrap.cc:492] T 2125fe6835ea47478b7206b78a54c11a P 8a926d64ca6f4488bdeef5143e6675d7: No bootstrap required, opened a new log
I20260807 18:37:14.818537 30957 ts_tablet_manager.cc:1397] T 2125fe6835ea47478b7206b78a54c11a P 8a926d64ca6f4488bdeef5143e6675d7: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260807 18:37:14.819653 30961 tablet_bootstrap.cc:492] T 2125fe6835ea47478b7206b78a54c11a P 498232f6bbe549a7a8284e3c51f05d02: No bootstrap required, opened a new log
I20260807 18:37:14.819835 30959 tablet_bootstrap.cc:492] T 2125fe6835ea47478b7206b78a54c11a P fdff3b07c7cf4746a694ab2ddff2ec63: No bootstrap required, opened a new log
I20260807 18:37:14.819852 30961 ts_tablet_manager.cc:1397] T 2125fe6835ea47478b7206b78a54c11a P 498232f6bbe549a7a8284e3c51f05d02: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260807 18:37:14.820007 30959 ts_tablet_manager.cc:1397] T 2125fe6835ea47478b7206b78a54c11a P fdff3b07c7cf4746a694ab2ddff2ec63: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260807 18:37:14.823144 30957 raft_consensus.cc:348] T 2125fe6835ea47478b7206b78a54c11a P 8a926d64ca6f4488bdeef5143e6675d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a926d64ca6f4488bdeef5143e6675d7" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 37977 } } peers { permanent_uuid: "fdff3b07c7cf4746a694ab2ddff2ec63" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 46339 } } peers { permanent_uuid: "498232f6bbe549a7a8284e3c51f05d02" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 39787 } }
I20260807 18:37:14.823130 30961 raft_consensus.cc:348] T 2125fe6835ea47478b7206b78a54c11a P 498232f6bbe549a7a8284e3c51f05d02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a926d64ca6f4488bdeef5143e6675d7" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 37977 } } peers { permanent_uuid: "fdff3b07c7cf4746a694ab2ddff2ec63" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 46339 } } peers { permanent_uuid: "498232f6bbe549a7a8284e3c51f05d02" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 39787 } }
I20260807 18:37:14.823431 30957 raft_consensus.cc:374] T 2125fe6835ea47478b7206b78a54c11a P 8a926d64ca6f4488bdeef5143e6675d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:37:14.823459 30961 raft_consensus.cc:374] T 2125fe6835ea47478b7206b78a54c11a P 498232f6bbe549a7a8284e3c51f05d02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:37:14.823478 30957 raft_consensus.cc:729] T 2125fe6835ea47478b7206b78a54c11a P 8a926d64ca6f4488bdeef5143e6675d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a926d64ca6f4488bdeef5143e6675d7, State: Initialized, Role: FOLLOWER
I20260807 18:37:14.823513 30961 raft_consensus.cc:729] T 2125fe6835ea47478b7206b78a54c11a P 498232f6bbe549a7a8284e3c51f05d02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 498232f6bbe549a7a8284e3c51f05d02, State: Initialized, Role: FOLLOWER
I20260807 18:37:14.823977 30957 consensus_queue.cc:260] T 2125fe6835ea47478b7206b78a54c11a P 8a926d64ca6f4488bdeef5143e6675d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a926d64ca6f4488bdeef5143e6675d7" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 37977 } } peers { permanent_uuid: "fdff3b07c7cf4746a694ab2ddff2ec63" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 46339 } } peers { permanent_uuid: "498232f6bbe549a7a8284e3c51f05d02" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 39787 } }
I20260807 18:37:14.824010 30961 consensus_queue.cc:260] T 2125fe6835ea47478b7206b78a54c11a P 498232f6bbe549a7a8284e3c51f05d02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a926d64ca6f4488bdeef5143e6675d7" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 37977 } } peers { permanent_uuid: "fdff3b07c7cf4746a694ab2ddff2ec63" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 46339 } } peers { permanent_uuid: "498232f6bbe549a7a8284e3c51f05d02" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 39787 } }
I20260807 18:37:14.824980 30936 heartbeater.cc:503] Master 127.11.14.126:39329 was elected leader, sending a full tablet report...
I20260807 18:37:14.825069 30957 ts_tablet_manager.cc:1428] T 2125fe6835ea47478b7206b78a54c11a P 8a926d64ca6f4488bdeef5143e6675d7: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260807 18:37:14.825114 30961 ts_tablet_manager.cc:1428] T 2125fe6835ea47478b7206b78a54c11a P 498232f6bbe549a7a8284e3c51f05d02: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260807 18:37:14.825217 30959 raft_consensus.cc:348] T 2125fe6835ea47478b7206b78a54c11a P fdff3b07c7cf4746a694ab2ddff2ec63 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a926d64ca6f4488bdeef5143e6675d7" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 37977 } } peers { permanent_uuid: "fdff3b07c7cf4746a694ab2ddff2ec63" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 46339 } } peers { permanent_uuid: "498232f6bbe549a7a8284e3c51f05d02" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 39787 } }
I20260807 18:37:14.825562 30959 raft_consensus.cc:374] T 2125fe6835ea47478b7206b78a54c11a P fdff3b07c7cf4746a694ab2ddff2ec63 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:37:14.825623 30645 heartbeater.cc:503] Master 127.11.14.126:39329 was elected leader, sending a full tablet report...
I20260807 18:37:14.825654 30959 raft_consensus.cc:729] T 2125fe6835ea47478b7206b78a54c11a P fdff3b07c7cf4746a694ab2ddff2ec63 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fdff3b07c7cf4746a694ab2ddff2ec63, State: Initialized, Role: FOLLOWER
I20260807 18:37:14.826090 30959 consensus_queue.cc:260] T 2125fe6835ea47478b7206b78a54c11a P fdff3b07c7cf4746a694ab2ddff2ec63 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a926d64ca6f4488bdeef5143e6675d7" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 37977 } } peers { permanent_uuid: "fdff3b07c7cf4746a694ab2ddff2ec63" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 46339 } } peers { permanent_uuid: "498232f6bbe549a7a8284e3c51f05d02" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 39787 } }
I20260807 18:37:14.827047 30959 ts_tablet_manager.cc:1428] T 2125fe6835ea47478b7206b78a54c11a P fdff3b07c7cf4746a694ab2ddff2ec63: Time spent starting tablet: real 0.007s user 0.002s sys 0.004s
I20260807 18:37:14.827075 30792 heartbeater.cc:503] Master 127.11.14.126:39329 was elected leader, sending a full tablet report...
W20260807 18:37:14.848062 30646 tablet.cc:2369] T 2125fe6835ea47478b7206b78a54c11a P 498232f6bbe549a7a8284e3c51f05d02: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:37:14.977495 30966 raft_consensus.cc:482] T 2125fe6835ea47478b7206b78a54c11a P 8a926d64ca6f4488bdeef5143e6675d7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:37:14.977780 30966 raft_consensus.cc:504] T 2125fe6835ea47478b7206b78a54c11a P 8a926d64ca6f4488bdeef5143e6675d7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a926d64ca6f4488bdeef5143e6675d7" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 37977 } } peers { permanent_uuid: "fdff3b07c7cf4746a694ab2ddff2ec63" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 46339 } } peers { permanent_uuid: "498232f6bbe549a7a8284e3c51f05d02" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 39787 } }
I20260807 18:37:14.979121 30966 leader_election.cc:288] T 2125fe6835ea47478b7206b78a54c11a P 8a926d64ca6f4488bdeef5143e6675d7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fdff3b07c7cf4746a694ab2ddff2ec63 (127.11.14.66:46339), 498232f6bbe549a7a8284e3c51f05d02 (127.11.14.65:39787)
I20260807 18:37:14.983034 30939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:14.979423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.66:46339 (local address 127.11.14.67:33047)
0807 18:37:14.979590 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0807 18:37:14.979608 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 18:37:14.979737 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:37:14.980174 (+ 437us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:37:14.980179 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:37:14.980203 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:37:14.980345 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:37:14.980352 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:37:14.981097 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:37:14.981102 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:37:14.981923 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:37:14.981932 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:37:14.982588 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:37:14.982592 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:37:14.982692 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:14.982715 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 18:37:14.982791 (+ 76us) client_negotiation.cc:240] Negotiation successful
0807 18:37:14.982863 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49,"mutex_wait_us":1}
I20260807 18:37:14.983227 30955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:14.979526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.67:33047 (local address 127.11.14.66:46339)
0807 18:37:14.979695 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0807 18:37:14.979703 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:37:14.979724 (+ 21us) server_negotiation.cc:407] Connection header received
0807 18:37:14.979865 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:37:14.979867 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:37:14.979937 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:37:14.980036 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:37:14.980454 (+ 418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:14.980968 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:14.982061 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:14.982460 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:14.982594 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:14.982646 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0807 18:37:14.983006 (+ 360us) server_negotiation.cc:299] Negotiation successful
0807 18:37:14.983108 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260807 18:37:14.983269 30956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:14.980044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.67:56457 (local address 127.11.14.65:39787)
0807 18:37:14.980185 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0807 18:37:14.980188 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 18:37:14.980223 (+ 35us) server_negotiation.cc:407] Connection header received
0807 18:37:14.980338 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:37:14.980340 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:37:14.980378 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:37:14.980486 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:37:14.980771 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:14.981249 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:14.982268 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:14.982659 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:14.982793 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:14.982838 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0807 18:37:14.983042 (+ 204us) server_negotiation.cc:299] Negotiation successful
0807 18:37:14.983137 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260807 18:37:14.983649 30969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:14.979750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.65:39787 (local address 127.11.14.67:56457)
0807 18:37:14.980149 (+ 399us) negotiation.cc:107] Waiting for socket to connect
0807 18:37:14.980163 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 18:37:14.980253 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:37:14.980500 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:37:14.980503 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:37:14.980520 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:37:14.980650 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:37:14.980655 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:37:14.981365 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:37:14.981371 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:37:14.982143 (+ 772us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:37:14.982152 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:37:14.982801 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:37:14.982804 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:37:14.982894 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:14.982915 (+ 21us) client_negotiation.cc:769] Sending connection context
0807 18:37:14.983391 (+ 476us) client_negotiation.cc:240] Negotiation successful
0807 18:37:14.983500 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":324,"mutex_wait_us":25,"thread_start_us":144,"threads_started":1}
I20260807 18:37:14.983899 30742 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2125fe6835ea47478b7206b78a54c11a" candidate_uuid: "8a926d64ca6f4488bdeef5143e6675d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fdff3b07c7cf4746a694ab2ddff2ec63" is_pre_election: true
I20260807 18:37:14.984277 30742 raft_consensus.cc:2454] T 2125fe6835ea47478b7206b78a54c11a P fdff3b07c7cf4746a694ab2ddff2ec63 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a926d64ca6f4488bdeef5143e6675d7 in term 0.
I20260807 18:37:14.984570 30593 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2125fe6835ea47478b7206b78a54c11a" candidate_uuid: "8a926d64ca6f4488bdeef5143e6675d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "498232f6bbe549a7a8284e3c51f05d02" is_pre_election: true
I20260807 18:37:14.984824 30814 leader_election.cc:302] T 2125fe6835ea47478b7206b78a54c11a P 8a926d64ca6f4488bdeef5143e6675d7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a926d64ca6f4488bdeef5143e6675d7, fdff3b07c7cf4746a694ab2ddff2ec63; no voters:
I20260807 18:37:14.984879 30593 raft_consensus.cc:2454] T 2125fe6835ea47478b7206b78a54c11a P 498232f6bbe549a7a8284e3c51f05d02 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a926d64ca6f4488bdeef5143e6675d7 in term 0.
I20260807 18:37:14.985140 30966 raft_consensus.cc:2781] T 2125fe6835ea47478b7206b78a54c11a P 8a926d64ca6f4488bdeef5143e6675d7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:37:14.985262 30966 raft_consensus.cc:482] T 2125fe6835ea47478b7206b78a54c11a P 8a926d64ca6f4488bdeef5143e6675d7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:37:14.985337 30966 raft_consensus.cc:3037] T 2125fe6835ea47478b7206b78a54c11a P 8a926d64ca6f4488bdeef5143e6675d7 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:37:14.986416 30966 raft_consensus.cc:504] T 2125fe6835ea47478b7206b78a54c11a P 8a926d64ca6f4488bdeef5143e6675d7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a926d64ca6f4488bdeef5143e6675d7" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 37977 } } peers { permanent_uuid: "fdff3b07c7cf4746a694ab2ddff2ec63" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 46339 } } peers { permanent_uuid: "498232f6bbe549a7a8284e3c51f05d02" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 39787 } }
I20260807 18:37:14.986878 30966 leader_election.cc:288] T 2125fe6835ea47478b7206b78a54c11a P 8a926d64ca6f4488bdeef5143e6675d7 [CANDIDATE]: Term 1 election: Requested vote from peers fdff3b07c7cf4746a694ab2ddff2ec63 (127.11.14.66:46339), 498232f6bbe549a7a8284e3c51f05d02 (127.11.14.65:39787)
I20260807 18:37:14.987311 30742 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2125fe6835ea47478b7206b78a54c11a" candidate_uuid: "8a926d64ca6f4488bdeef5143e6675d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fdff3b07c7cf4746a694ab2ddff2ec63"
I20260807 18:37:14.987342 30593 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2125fe6835ea47478b7206b78a54c11a" candidate_uuid: "8a926d64ca6f4488bdeef5143e6675d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "498232f6bbe549a7a8284e3c51f05d02"
I20260807 18:37:14.987453 30742 raft_consensus.cc:3037] T 2125fe6835ea47478b7206b78a54c11a P fdff3b07c7cf4746a694ab2ddff2ec63 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:37:14.987478 30593 raft_consensus.cc:3037] T 2125fe6835ea47478b7206b78a54c11a P 498232f6bbe549a7a8284e3c51f05d02 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:37:14.988569 30742 raft_consensus.cc:2454] T 2125fe6835ea47478b7206b78a54c11a P fdff3b07c7cf4746a694ab2ddff2ec63 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a926d64ca6f4488bdeef5143e6675d7 in term 1.
I20260807 18:37:14.988605 30593 raft_consensus.cc:2454] T 2125fe6835ea47478b7206b78a54c11a P 498232f6bbe549a7a8284e3c51f05d02 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a926d64ca6f4488bdeef5143e6675d7 in term 1.
I20260807 18:37:14.988909 30814 leader_election.cc:302] T 2125fe6835ea47478b7206b78a54c11a P 8a926d64ca6f4488bdeef5143e6675d7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a926d64ca6f4488bdeef5143e6675d7, fdff3b07c7cf4746a694ab2ddff2ec63; no voters:
I20260807 18:37:14.989089 30966 raft_consensus.cc:2781] T 2125fe6835ea47478b7206b78a54c11a P 8a926d64ca6f4488bdeef5143e6675d7 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:37:14.989372 30966 raft_consensus.cc:686] T 2125fe6835ea47478b7206b78a54c11a P 8a926d64ca6f4488bdeef5143e6675d7 [term 1 LEADER]: Becoming Leader. State: Replica: 8a926d64ca6f4488bdeef5143e6675d7, State: Running, Role: LEADER
I20260807 18:37:14.989753 30966 consensus_queue.cc:237] T 2125fe6835ea47478b7206b78a54c11a P 8a926d64ca6f4488bdeef5143e6675d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a926d64ca6f4488bdeef5143e6675d7" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 37977 } } peers { permanent_uuid: "fdff3b07c7cf4746a694ab2ddff2ec63" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 46339 } } peers { permanent_uuid: "498232f6bbe549a7a8284e3c51f05d02" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 39787 } }
I20260807 18:37:14.993266 30431 catalog_manager.cc:5495] T 2125fe6835ea47478b7206b78a54c11a P 8a926d64ca6f4488bdeef5143e6675d7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8a926d64ca6f4488bdeef5143e6675d7 (127.11.14.67). New cstate: current_term: 1 leader_uuid: "8a926d64ca6f4488bdeef5143e6675d7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a926d64ca6f4488bdeef5143e6675d7" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 37977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fdff3b07c7cf4746a694ab2ddff2ec63" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 46339 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "498232f6bbe549a7a8284e3c51f05d02" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 39787 } health_report { overall_health: UNKNOWN } } }
W20260807 18:37:15.002380 30937 tablet.cc:2369] T 2125fe6835ea47478b7206b78a54c11a P 8a926d64ca6f4488bdeef5143e6675d7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:37:15.037858 30952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:15.034508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53620 (local address 127.11.14.67:37977)
0807 18:37:15.034639 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0807 18:37:15.034643 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:37:15.034659 (+ 16us) server_negotiation.cc:407] Connection header received
0807 18:37:15.034766 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:37:15.034777 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:37:15.034837 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:37:15.034928 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:37:15.035158 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:15.035738 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:15.036567 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:15.036769 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:15.036844 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:15.037107 (+ 263us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:37:15.037546 (+ 439us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:37:15.037636 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0807 18:37:15.037689 (+ 53us) server_negotiation.cc:299] Negotiation successful
0807 18:37:15.037732 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260807 18:37:15.043917 30742 raft_consensus.cc:1261] T 2125fe6835ea47478b7206b78a54c11a P fdff3b07c7cf4746a694ab2ddff2ec63 [term 1 FOLLOWER]: Refusing update from remote peer 8a926d64ca6f4488bdeef5143e6675d7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:37:15.043943 30593 raft_consensus.cc:1261] T 2125fe6835ea47478b7206b78a54c11a P 498232f6bbe549a7a8284e3c51f05d02 [term 1 FOLLOWER]: Refusing update from remote peer 8a926d64ca6f4488bdeef5143e6675d7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:37:15.044689 30966 consensus_queue.cc:1035] T 2125fe6835ea47478b7206b78a54c11a P 8a926d64ca6f4488bdeef5143e6675d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fdff3b07c7cf4746a694ab2ddff2ec63" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 46339 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:37:15.044889 30970 consensus_queue.cc:1035] T 2125fe6835ea47478b7206b78a54c11a P 8a926d64ca6f4488bdeef5143e6675d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "498232f6bbe549a7a8284e3c51f05d02" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 39787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:37:15.062678 30430 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:54478:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:37:15.062852 30430 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:54478:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:37:15.065375 30430 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 812d2d59f64645b68893671f6317fccb: Sending DeleteTablet for 3 replicas of tablet 2125fe6835ea47478b7206b78a54c11a
I20260807 18:37:15.066218 30722 tablet_service.cc:1514] Processing DeleteTablet for tablet 2125fe6835ea47478b7206b78a54c11a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:37:15 UTC) from {username='slave'} at 127.0.0.1:50710
I20260807 18:37:15.066322 30571 tablet_service.cc:1514] Processing DeleteTablet for tablet 2125fe6835ea47478b7206b78a54c11a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:37:15 UTC) from {username='slave'} at 127.0.0.1:40530
I20260807 18:37:15.067087 30864 tablet_service.cc:1514] Processing DeleteTablet for tablet 2125fe6835ea47478b7206b78a54c11a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:37:15 UTC) from {username='slave'} at 127.0.0.1:53614
I20260807 18:37:15.067533 30988 tablet_replica.cc:330] T 2125fe6835ea47478b7206b78a54c11a P 8a926d64ca6f4488bdeef5143e6675d7: stopping tablet replica
I20260807 18:37:15.067909 30988 raft_consensus.cc:2229] T 2125fe6835ea47478b7206b78a54c11a P 8a926d64ca6f4488bdeef5143e6675d7 [term 1 LEADER]: Raft consensus shutting down.
I20260807 18:37:15.067977 11321 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkz56XN/build/debug/bin/kudu with pid 30497
I20260807 18:37:15.068295 30988 raft_consensus.cc:2258] T 2125fe6835ea47478b7206b78a54c11a P 8a926d64ca6f4488bdeef5143e6675d7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:37:15.071030 30988 ts_tablet_manager.cc:1905] T 2125fe6835ea47478b7206b78a54c11a P 8a926d64ca6f4488bdeef5143e6675d7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:37:15.073501 30988 ts_tablet_manager.cc:1918] T 2125fe6835ea47478b7206b78a54c11a P 8a926d64ca6f4488bdeef5143e6675d7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:37:15.073626 30988 log.cc:1196] T 2125fe6835ea47478b7206b78a54c11a P 8a926d64ca6f4488bdeef5143e6675d7: Deleting WAL directory at /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786127579900523-11321-0/minicluster-data/ts-2/wal/wals/2125fe6835ea47478b7206b78a54c11a
I20260807 18:37:15.073928 30988 ts_tablet_manager.cc:1939] T 2125fe6835ea47478b7206b78a54c11a P 8a926d64ca6f4488bdeef5143e6675d7: Deleting consensus metadata
I20260807 18:37:15.075078 30416 catalog_manager.cc:4841] TS 8a926d64ca6f4488bdeef5143e6675d7 (127.11.14.67:37977): tablet 2125fe6835ea47478b7206b78a54c11a (table test-table [id=73b5edcbf9f14cf5851d0576f1def04c]) successfully deleted
I20260807 18:37:15.075763 30986 tablet_replica.cc:330] T 2125fe6835ea47478b7206b78a54c11a P fdff3b07c7cf4746a694ab2ddff2ec63: stopping tablet replica
I20260807 18:37:15.076114 30986 raft_consensus.cc:2229] T 2125fe6835ea47478b7206b78a54c11a P fdff3b07c7cf4746a694ab2ddff2ec63 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:37:15.076442 30986 raft_consensus.cc:2258] T 2125fe6835ea47478b7206b78a54c11a P fdff3b07c7cf4746a694ab2ddff2ec63 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:37:15.077718 30986 ts_tablet_manager.cc:1905] T 2125fe6835ea47478b7206b78a54c11a P fdff3b07c7cf4746a694ab2ddff2ec63: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260807 18:37:15.079988 30415 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.14.65:39787 (error 108)
I20260807 18:37:15.080210 11321 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkz56XN/build/debug/bin/kudu with pid 30648
I20260807 18:37:15.080585 30986 ts_tablet_manager.cc:1918] T 2125fe6835ea47478b7206b78a54c11a P fdff3b07c7cf4746a694ab2ddff2ec63: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:37:15.088447 30953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:15.088131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.65:39787 (local address 127.0.0.1:40546)
0807 18:37:15.088272 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0807 18:37:15.088365 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.65:39787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260807 18:37:15.088680 30415 catalog_manager.cc:4584] TS 498232f6bbe549a7a8284e3c51f05d02 (127.11.14.65:39787): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2125fe6835ea47478b7206b78a54c11a: Network error: Client connection negotiation failed: client connection to 127.11.14.65:39787: connect: Connection refused (error 111)
I20260807 18:37:15.089843 30953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:15.089597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.66:46339 (local address 127.0.0.1:50714)
0807 18:37:15.089707 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0807 18:37:15.089758 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.66:46339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260807 18:37:15.089886 11321 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkz56XN/build/debug/bin/kudu with pid 30796
I20260807 18:37:15.098886 11321 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkz56XN/build/debug/bin/kudu with pid 30397
2026-08-07T18:37:15Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (1257 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127579900523-11321-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:37:15 dist-test-slave-6k22 krb5kdc[30992](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:37:15 dist-test-slave-6k22 krb5kdc[30992](info): set up 2 sockets
Aug 07 18:37:15 dist-test-slave-6k22 krb5kdc[30992](info): commencing operation
krb5kdc: starting...
W20260807 18:37:16.200963 11321 mini_kdc.cc:121] Time spent starting KDC: real 1.078s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127579900523-11321-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127579900523-11321-0/krb5kdc/test-user.keytab.
Aug 07 18:37:16 dist-test-slave-6k22 krb5kdc[30992](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127836, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:37:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:37:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.14.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.14.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.14.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.126.keytab.
Entry for principal kudu/127.11.14.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.126.keytab.
I20260807 18:37:16.332121 11321 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkz56XN/build/debug/bin/kudu
/tmp/dist-test-taskkz56XN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127579900523-11321-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127579900523-11321-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127579900523-11321-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127579900523-11321-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.14.126:38513
--webserver_interface=127.11.14.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.126.keytab
--principal=kudu/127.11.14.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.14.84:44599
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.14.126:38513
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127579900523-11321-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127579900523-11321-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127579900523-11321-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:37:16.445043 31006 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:37:16.445354 31006 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:37:16.445438 31006 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:37:16.449404 31006 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:37:16.449489 31006 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:37:16.449529 31006 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:37:16.449566 31006 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:37:16.449601 31006 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:37:16.454779 31006 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.14.84:44599
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127579900523-11321-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127579900523-11321-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.14.126:38513
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.126.keytab
--principal=kudu/127.11.14.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.14.126:38513
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127579900523-11321-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.14.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31006
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127579900523-11321-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:37:16.455834 31006 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:37:16.456921 31006 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:37:16.465649 31014 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:37:16.465633 31013 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:37:16.465633 31016 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:37:16.465728 31006 server_base.cc:1034] running on GCE node
I20260807 18:37:16.466486 31006 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:37:16.467672 31006 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:37:16.468863 31006 hybrid_clock.cc:648] HybridClock initialized: now 1786127836468829 us; error 58 us; skew 500 ppm
Aug 07 18:37:16 dist-test-slave-6k22 krb5kdc[30992](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127836, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:37:16.478459 31006 init.cc:377] Logged in from keytab as kudu/127.11.14.126@KRBTEST.COM (short username kudu)
I20260807 18:37:16.479480 31006 webserver.cc:460] Webserver started at http://127.11.14.126:37677/ using document root <none> and password file <none>
I20260807 18:37:16.480125 31006 fs_manager.cc:359] Metadata directory not provided
I20260807 18:37:16.480191 31006 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:37:16.480434 31006 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:37:16.482561 31006 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127579900523-11321-0/minicluster-data/master-0/data/instance:
uuid: "b03547278cc84705bb7578776cb192d5"
format_stamp: "Formatted at 2026-08-07 18:37:16 on dist-test-slave-6k22"
server_key: "b3d15071aef5fb56470a07434842231f"
server_key_iv: "983908d5add778a68eb49500d10ae4b2"
server_key_version: "encryptionkey@0"
I20260807 18:37:16.483152 31006 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127579900523-11321-0/minicluster-data/master-0/wal/instance:
uuid: "b03547278cc84705bb7578776cb192d5"
format_stamp: "Formatted at 2026-08-07 18:37:16 on dist-test-slave-6k22"
server_key: "b3d15071aef5fb56470a07434842231f"
server_key_iv: "983908d5add778a68eb49500d10ae4b2"
server_key_version: "encryptionkey@0"
I20260807 18:37:16.487035 31006 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 18:37:16.489581 31025 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:37:16.490787 31006 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 18:37:16.490952 31006 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127579900523-11321-0/minicluster-data/master-0/data,/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127579900523-11321-0/minicluster-data/master-0/wal
uuid: "b03547278cc84705bb7578776cb192d5"
format_stamp: "Formatted at 2026-08-07 18:37:16 on dist-test-slave-6k22"
server_key: "b3d15071aef5fb56470a07434842231f"
server_key_iv: "983908d5add778a68eb49500d10ae4b2"
server_key_version: "encryptionkey@0"
I20260807 18:37:16.491078 31006 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127579900523-11321-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127579900523-11321-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127579900523-11321-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:37:16.500350 31006 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:37:16.503291 31006 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:37:16.503475 31006 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:37:16.510941 31006 rpc_server.cc:307] RPC server started. Bound to: 127.11.14.126:38513
I20260807 18:37:16.510968 31082 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.14.126:38513 every 8 connection(s)
I20260807 18:37:16.511919 31006 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127579900523-11321-0/minicluster-data/master-0/data/info.pb
I20260807 18:37:16.514619 31083 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:37:16.518955 11321 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkz56XN/build/debug/bin/kudu as pid 31006
I20260807 18:37:16.519106 11321 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127579900523-11321-0/minicluster-data/master-0/wal/instance
I20260807 18:37:16.519416 11321 external_mini_cluster.cc:1428] Setting key 99fb7a5b84dfd17c6d202d6962680935
I20260807 18:37:16.521078 31083 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b03547278cc84705bb7578776cb192d5: Bootstrap starting.
I20260807 18:37:16.523509 31083 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b03547278cc84705bb7578776cb192d5: Neither blocks nor log segments found. Creating new log.
I20260807 18:37:16.524294 31083 log.cc:824] T 00000000000000000000000000000000 P b03547278cc84705bb7578776cb192d5: Log is configured to *not* fsync() on all Append() calls
I20260807 18:37:16.526314 31083 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b03547278cc84705bb7578776cb192d5: No bootstrap required, opened a new log
Aug 07 18:37:16 dist-test-slave-6k22 krb5kdc[30992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127836, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.14.126@KRBTEST.COM
I20260807 18:37:16.530730 31083 raft_consensus.cc:348] T 00000000000000000000000000000000 P b03547278cc84705bb7578776cb192d5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b03547278cc84705bb7578776cb192d5" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 38513 } }
I20260807 18:37:16.530968 31083 raft_consensus.cc:374] T 00000000000000000000000000000000 P b03547278cc84705bb7578776cb192d5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:37:16.531044 31083 raft_consensus.cc:729] T 00000000000000000000000000000000 P b03547278cc84705bb7578776cb192d5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b03547278cc84705bb7578776cb192d5, State: Initialized, Role: FOLLOWER
I20260807 18:37:16.531538 31083 consensus_queue.cc:260] T 00000000000000000000000000000000 P b03547278cc84705bb7578776cb192d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b03547278cc84705bb7578776cb192d5" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 38513 } }
I20260807 18:37:16.531780 31083 raft_consensus.cc:388] T 00000000000000000000000000000000 P b03547278cc84705bb7578776cb192d5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:37:16.531860 31083 raft_consensus.cc:482] T 00000000000000000000000000000000 P b03547278cc84705bb7578776cb192d5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:37:16.531970 31083 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b03547278cc84705bb7578776cb192d5 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:37:16.533053 31083 raft_consensus.cc:504] T 00000000000000000000000000000000 P b03547278cc84705bb7578776cb192d5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b03547278cc84705bb7578776cb192d5" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 38513 } }
I20260807 18:37:16.533438 31083 leader_election.cc:302] T 00000000000000000000000000000000 P b03547278cc84705bb7578776cb192d5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b03547278cc84705bb7578776cb192d5; no voters:
I20260807 18:37:16.533799 31083 leader_election.cc:288] T 00000000000000000000000000000000 P b03547278cc84705bb7578776cb192d5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:37:16.533903 31088 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b03547278cc84705bb7578776cb192d5 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:37:16.534123 31088 raft_consensus.cc:686] T 00000000000000000000000000000000 P b03547278cc84705bb7578776cb192d5 [term 1 LEADER]: Becoming Leader. State: Replica: b03547278cc84705bb7578776cb192d5, State: Running, Role: LEADER
I20260807 18:37:16.534502 31088 consensus_queue.cc:237] T 00000000000000000000000000000000 P b03547278cc84705bb7578776cb192d5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b03547278cc84705bb7578776cb192d5" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 38513 } }
I20260807 18:37:16.534732 31086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:16.521007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48368 (local address 127.11.14.126:38513)
0807 18:37:16.521471 (+ 464us) server_negotiation.cc:206] Beginning negotiation
0807 18:37:16.521486 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 18:37:16.521523 (+ 37us) server_negotiation.cc:407] Connection header received
0807 18:37:16.522221 (+ 698us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:37:16.522239 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:37:16.522583 (+ 344us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:37:16.522935 (+ 352us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:37:16.523888 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:16.525172 (+ 1284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:16.526041 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:16.526592 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:16.526769 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:16.529682 (+ 2913us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:37:16.529687 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:37:16.529701 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:37:16.529728 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:37:16.531504 (+ 1776us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:37:16.532095 (+ 591us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:37:16.532102 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:37:16.532107 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:37:16.532232 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:37:16.532519 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:37:16.532521 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:37:16.532522 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:37:16.532894 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:37:16.533034 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0807 18:37:16.533378 (+ 344us) server_negotiation.cc:299] Negotiation successful
0807 18:37:16.533679 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":318,"thread_start_us":160,"threads_started":1}
I20260807 18:37:16.535154 31083 sys_catalog.cc:564] T 00000000000000000000000000000000 P b03547278cc84705bb7578776cb192d5 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:37:16.536195 31089 sys_catalog.cc:455] T 00000000000000000000000000000000 P b03547278cc84705bb7578776cb192d5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b03547278cc84705bb7578776cb192d5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b03547278cc84705bb7578776cb192d5" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 38513 } } }
I20260807 18:37:16.536370 31089 sys_catalog.cc:458] T 00000000000000000000000000000000 P b03547278cc84705bb7578776cb192d5 [sys.catalog]: This master's current role is: LEADER
I20260807 18:37:16.536623 31089 sys_catalog.cc:455] T 00000000000000000000000000000000 P b03547278cc84705bb7578776cb192d5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b03547278cc84705bb7578776cb192d5. Latest consensus state: current_term: 1 leader_uuid: "b03547278cc84705bb7578776cb192d5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b03547278cc84705bb7578776cb192d5" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 38513 } } }
I20260807 18:37:16.536731 31089 sys_catalog.cc:458] T 00000000000000000000000000000000 P b03547278cc84705bb7578776cb192d5 [sys.catalog]: This master's current role is: LEADER
I20260807 18:37:16.537973 31098 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:37:16.540896 31098 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:37:16.546842 31098 catalog_manager.cc:1347] Generated new cluster ID: c28ebe33c5d6449e88c6cb0a43ecd599
I20260807 18:37:16.547293 31098 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:37:16.579464 31098 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:37:16.580857 31098 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:37:16.587215 31098 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b03547278cc84705bb7578776cb192d5: Generated new TSK 0
I20260807 18:37:16.588068 31098 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.14.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.14.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.14.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.65.keytab.
Entry for principal kudu/127.11.14.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.65.keytab.
I20260807 18:37:16.624433 11321 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkz56XN/build/debug/bin/kudu
/tmp/dist-test-taskkz56XN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127579900523-11321-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127579900523-11321-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127579900523-11321-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127579900523-11321-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.14.65:0
--local_ip_for_outbound_sockets=127.11.14.65
--webserver_interface=127.11.14.65
--webserver_port=0
--tserver_master_addrs=127.11.14.126:38513
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.65.keytab
--principal=kudu/127.11.14.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.14.84:44599
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127579900523-11321-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127579900523-11321-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127579900523-11321-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:37:16.741923 31111 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:37:16.742182 31111 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:37:16.742259 31111 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:37:16.746284 31111 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:37:16.746344 31111 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:37:16.746459 31111 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.14.65
I20260807 18:37:16.751674 31111 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.14.84:44599
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127579900523-11321-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127579900523-11321-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.65.keytab
--principal=kudu/127.11.14.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.14.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127579900523-11321-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.14.65
--webserver_port=0
--tserver_master_addrs=127.11.14.126:38513
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31111
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.14.65
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127579900523-11321-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:37:16.752717 31111 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:37:16.753798 31111 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:37:16.763763 31118 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:37:16.763774 31117 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:37:16.763774 31111 server_base.cc:1034] running on GCE node
W20260807 18:37:16.763795 31120 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:37:16.764395 31111 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:37:16.765009 31111 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:37:16.766197 31111 hybrid_clock.cc:648] HybridClock initialized: now 1786127836766181 us; error 27 us; skew 500 ppm
Aug 07 18:37:16 dist-test-slave-6k22 krb5kdc[30992](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127836, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:37:16.768977 31111 init.cc:377] Logged in from keytab as kudu/127.11.14.65@KRBTEST.COM (short username kudu)
I20260807 18:37:16.769845 31111 webserver.cc:460] Webserver started at http://127.11.14.65:42965/ using document root <none> and password file <none>
I20260807 18:37:16.770453 31111 fs_manager.cc:359] Metadata directory not provided
I20260807 18:37:16.770545 31111 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:37:16.770781 31111 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:37:16.772907 31111 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127579900523-11321-0/minicluster-data/ts-0/data/instance:
uuid: "39cfa69c502e42b295bb8d2c4b984f70"
format_stamp: "Formatted at 2026-08-07 18:37:16 on dist-test-slave-6k22"
server_key: "aac3ced5787084080c10a9d73e6b69ad"
server_key_iv: "d4249b0297bbb22da2b1d4a4b140a88d"
server_key_version: "encryptionkey@0"
I20260807 18:37:16.773496 31111 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127579900523-11321-0/minicluster-data/ts-0/wal/instance:
uuid: "39cfa69c502e42b295bb8d2c4b984f70"
format_stamp: "Formatted at 2026-08-07 18:37:16 on dist-test-slave-6k22"
server_key: "aac3ced5787084080c10a9d73e6b69ad"
server_key_iv: "d4249b0297bbb22da2b1d4a4b140a88d"
server_key_version: "encryptionkey@0"
I20260807 18:37:16.777856 31111 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.001s
I20260807 18:37:16.781451 31127 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:37:16.783216 31111 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 18:37:16.783407 31111 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127579900523-11321-0/minicluster-data/ts-0/data,/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127579900523-11321-0/minicluster-data/ts-0/wal
uuid: "39cfa69c502e42b295bb8d2c4b984f70"
format_stamp: "Formatted at 2026-08-07 18:37:16 on dist-test-slave-6k22"
server_key: "aac3ced5787084080c10a9d73e6b69ad"
server_key_iv: "d4249b0297bbb22da2b1d4a4b140a88d"
server_key_version: "encryptionkey@0"
I20260807 18:37:16.783638 31111 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127579900523-11321-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127579900523-11321-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127579900523-11321-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:37:16.812955 31111 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:37:16.815840 31111 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:37:16.815994 31111 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:37:16.816536 31111 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:37:16.817535 31111 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:37:16.817606 31111 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:37:16.817674 31111 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:37:16.817706 31111 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:37:16.827402 31111 rpc_server.cc:307] RPC server started. Bound to: 127.11.14.65:40479
I20260807 18:37:16.827502 31253 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.14.65:40479 every 8 connection(s)
I20260807 18:37:16.828387 31111 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127579900523-11321-0/minicluster-data/ts-0/data/info.pb
I20260807 18:37:16.831847 11321 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkz56XN/build/debug/bin/kudu as pid 31111
I20260807 18:37:16.832036 11321 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127579900523-11321-0/minicluster-data/ts-0/wal/instance
I20260807 18:37:16.832523 11321 external_mini_cluster.cc:1428] Setting key 80e9e4ff525aae22263a83fd14414387
Aug 07 18:37:16 dist-test-slave-6k22 krb5kdc[30992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127836, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.65@KRBTEST.COM for kudu/127.11.14.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.11.14.66@KRBTEST.COM; defaulting to no policy
I20260807 18:37:16.845427 31086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:16.830330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.65:36075 (local address 127.11.14.126:38513)
0807 18:37:16.830481 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0807 18:37:16.830486 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:37:16.831429 (+ 943us) server_negotiation.cc:407] Connection header received
0807 18:37:16.833453 (+ 2024us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:37:16.833456 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:37:16.833500 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:37:16.833557 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:37:16.835322 (+ 1765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:16.836088 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:16.837825 (+ 1737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:16.838122 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:16.838265 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:16.841299 (+ 3034us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:37:16.841304 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:37:16.841307 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:37:16.841338 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:37:16.842990 (+ 1652us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:37:16.843718 (+ 728us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:37:16.843723 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:37:16.843728 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:37:16.843834 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:37:16.844325 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:37:16.844327 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:37:16.844328 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:37:16.844574 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:37:16.844652 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0807 18:37:16.845161 (+ 509us) server_negotiation.cc:299] Negotiation successful
0807 18:37:16.845264 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.14.66@KRBTEST.COM" created.
I20260807 18:37:16.846462 31257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:16.830742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:38513 (local address 127.11.14.65:36075)
0807 18:37:16.831261 (+ 519us) negotiation.cc:107] Waiting for socket to connect
0807 18:37:16.831311 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0807 18:37:16.833216 (+ 1905us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:37:16.833787 (+ 571us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:37:16.833798 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:37:16.834348 (+ 550us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:37:16.835146 (+ 798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:37:16.835175 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:37:16.836228 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:37:16.836236 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:37:16.837671 (+ 1435us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:37:16.837683 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:37:16.838310 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:37:16.838315 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:37:16.838530 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:16.839282 (+ 752us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:37:16.839302 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:37:16.841105 (+ 1803us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:37:16.843171 (+ 2066us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:37:16.843179 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:37:16.843196 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:37:16.843561 (+ 365us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:37:16.844046 (+ 485us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:37:16.844048 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:37:16.844050 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:37:16.844198 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:37:16.844681 (+ 483us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:37:16.844688 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:37:16.844937 (+ 249us) client_negotiation.cc:769] Sending connection context
0807 18:37:16.845114 (+ 177us) client_negotiation.cc:240] Negotiation successful
0807 18:37:16.845408 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":298,"thread_start_us":118,"threads_started":1}
I20260807 18:37:16.848752 31255 heartbeater.cc:347] Connected to a master server at 127.11.14.126:38513
I20260807 18:37:16.849043 31255 heartbeater.cc:464] Registering TS with master...
I20260807 18:37:16.849548 31255 heartbeater.cc:511] Master 127.11.14.126:38513 requested a full tablet report, sending...
I20260807 18:37:16.851279 31045 ts_manager.cc:194] Registered new tserver with Master: 39cfa69c502e42b295bb8d2c4b984f70 (127.11.14.65:40479)
I20260807 18:37:16.853060 31045 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.14.65@KRBTEST.COM'} at 127.11.14.65:36075
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.14.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.66.keytab.
Entry for principal kudu/127.11.14.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.66.keytab.
I20260807 18:37:16.864595 11321 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkz56XN/build/debug/bin/kudu
/tmp/dist-test-taskkz56XN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127579900523-11321-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127579900523-11321-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127579900523-11321-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127579900523-11321-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.14.66:0
--local_ip_for_outbound_sockets=127.11.14.66
--webserver_interface=127.11.14.66
--webserver_port=0
--tserver_master_addrs=127.11.14.126:38513
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.66.keytab
--principal=kudu/127.11.14.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.14.84:44599
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127579900523-11321-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127579900523-11321-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127579900523-11321-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:37:16.982394 31260 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:37:16.982682 31260 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:37:16.982777 31260 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:37:16.986901 31260 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:37:16.986967 31260 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:37:16.987071 31260 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.14.66
I20260807 18:37:16.992579 31260 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.14.84:44599
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127579900523-11321-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127579900523-11321-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.66.keytab
--principal=kudu/127.11.14.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.14.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127579900523-11321-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.14.66
--webserver_port=0
--tserver_master_addrs=127.11.14.126:38513
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31260
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.14.66
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127579900523-11321-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:37:16.993667 31260 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:37:16.994735 31260 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:37:17.004993 31268 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:37:17.005056 31265 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:37:17.005259 31260 server_base.cc:1034] running on GCE node
W20260807 18:37:17.005174 31266 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:37:17.005872 31260 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:37:17.006605 31260 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:37:17.007835 31260 hybrid_clock.cc:648] HybridClock initialized: now 1786127837007798 us; error 46 us; skew 500 ppm
Aug 07 18:37:17 dist-test-slave-6k22 krb5kdc[30992](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127837, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:37:17.010620 31260 init.cc:377] Logged in from keytab as kudu/127.11.14.66@KRBTEST.COM (short username kudu)
I20260807 18:37:17.011523 31260 webserver.cc:460] Webserver started at http://127.11.14.66:46027/ using document root <none> and password file <none>
I20260807 18:37:17.012120 31260 fs_manager.cc:359] Metadata directory not provided
I20260807 18:37:17.012180 31260 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:37:17.012370 31260 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:37:17.014401 31260 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127579900523-11321-0/minicluster-data/ts-1/data/instance:
uuid: "5618b1865a3641509fd0aef10fa3790e"
format_stamp: "Formatted at 2026-08-07 18:37:17 on dist-test-slave-6k22"
server_key: "e92dac4f6adfc8b40df03d8db1c1ba29"
server_key_iv: "828137ca940b6b7e7515e636d1fe1ef1"
server_key_version: "encryptionkey@0"
I20260807 18:37:17.014892 31260 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127579900523-11321-0/minicluster-data/ts-1/wal/instance:
uuid: "5618b1865a3641509fd0aef10fa3790e"
format_stamp: "Formatted at 2026-08-07 18:37:17 on dist-test-slave-6k22"
server_key: "e92dac4f6adfc8b40df03d8db1c1ba29"
server_key_iv: "828137ca940b6b7e7515e636d1fe1ef1"
server_key_version: "encryptionkey@0"
I20260807 18:37:17.018589 31260 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.003s
I20260807 18:37:17.021166 31275 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:37:17.022490 31260 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 18:37:17.022634 31260 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127579900523-11321-0/minicluster-data/ts-1/data,/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127579900523-11321-0/minicluster-data/ts-1/wal
uuid: "5618b1865a3641509fd0aef10fa3790e"
format_stamp: "Formatted at 2026-08-07 18:37:17 on dist-test-slave-6k22"
server_key: "e92dac4f6adfc8b40df03d8db1c1ba29"
server_key_iv: "828137ca940b6b7e7515e636d1fe1ef1"
server_key_version: "encryptionkey@0"
I20260807 18:37:17.022746 31260 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127579900523-11321-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127579900523-11321-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127579900523-11321-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:37:17.039919 31260 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:37:17.043334 31260 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:37:17.043857 31260 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:37:17.044647 31260 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:37:17.045670 31260 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:37:17.045758 31260 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:37:17.045847 31260 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:37:17.045912 31260 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:37:17.056581 31260 rpc_server.cc:307] RPC server started. Bound to: 127.11.14.66:38239
I20260807 18:37:17.056605 31399 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.14.66:38239 every 8 connection(s)
I20260807 18:37:17.057639 31260 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127579900523-11321-0/minicluster-data/ts-1/data/info.pb
I20260807 18:37:17.061869 11321 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkz56XN/build/debug/bin/kudu as pid 31260
I20260807 18:37:17.062013 11321 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127579900523-11321-0/minicluster-data/ts-1/wal/instance
I20260807 18:37:17.062381 11321 external_mini_cluster.cc:1428] Setting key c307866540f5e29e27da17a79beb9003
Aug 07 18:37:17 dist-test-slave-6k22 krb5kdc[30992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127837, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.66@KRBTEST.COM for kudu/127.11.14.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.11.14.67@KRBTEST.COM; defaulting to no policy
I20260807 18:37:17.074234 31086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:17.059679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.66:35729 (local address 127.11.14.126:38513)
0807 18:37:17.059825 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0807 18:37:17.059829 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:37:17.061258 (+ 1429us) server_negotiation.cc:407] Connection header received
0807 18:37:17.062731 (+ 1473us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:37:17.062734 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:37:17.062781 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:37:17.062896 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:37:17.064551 (+ 1655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:17.065080 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:17.067018 (+ 1938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:17.067294 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:17.067433 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:17.070515 (+ 3082us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:37:17.070524 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:37:17.070526 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:37:17.070549 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:37:17.072082 (+ 1533us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:37:17.072613 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:37:17.072617 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:37:17.072619 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:37:17.072698 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:37:17.073063 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:37:17.073066 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:37:17.073068 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:37:17.073285 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:37:17.073409 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0807 18:37:17.073929 (+ 520us) server_negotiation.cc:299] Negotiation successful
0807 18:37:17.074056 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260807 18:37:17.075102 31402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:17.060134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:38513 (local address 127.11.14.66:35729)
0807 18:37:17.061074 (+ 940us) negotiation.cc:107] Waiting for socket to connect
0807 18:37:17.061132 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0807 18:37:17.062460 (+ 1328us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:37:17.063097 (+ 637us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:37:17.063109 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:37:17.063638 (+ 529us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:37:17.064368 (+ 730us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:37:17.064393 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:37:17.065234 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:37:17.065242 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:37:17.066867 (+ 1625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:37:17.066878 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:37:17.067475 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:37:17.067480 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:37:17.067739 (+ 259us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:17.068462 (+ 723us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:37:17.068483 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:37:17.070329 (+ 1846us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:37:17.072209 (+ 1880us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:37:17.072215 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:37:17.072230 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:37:17.072494 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:37:17.072791 (+ 297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:37:17.072810 (+ 19us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:37:17.072812 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:37:17.072957 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:37:17.073408 (+ 451us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:37:17.073416 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:37:17.073672 (+ 256us) client_negotiation.cc:769] Sending connection context
0807 18:37:17.073872 (+ 200us) client_negotiation.cc:240] Negotiation successful
0807 18:37:17.074247 (+ 375us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":720,"thread_start_us":555,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.14.67@KRBTEST.COM" created.
I20260807 18:37:17.076462 31400 heartbeater.cc:347] Connected to a master server at 127.11.14.126:38513
I20260807 18:37:17.076754 31400 heartbeater.cc:464] Registering TS with master...
I20260807 18:37:17.077399 31400 heartbeater.cc:511] Master 127.11.14.126:38513 requested a full tablet report, sending...
I20260807 18:37:17.078606 31045 ts_manager.cc:194] Registered new tserver with Master: 5618b1865a3641509fd0aef10fa3790e (127.11.14.66:38239)
I20260807 18:37:17.079244 31045 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.14.66@KRBTEST.COM'} at 127.11.14.66:35729
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.14.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.67.keytab.
Entry for principal kudu/127.11.14.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.67.keytab.
I20260807 18:37:17.094425 11321 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkz56XN/build/debug/bin/kudu
/tmp/dist-test-taskkz56XN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127579900523-11321-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127579900523-11321-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127579900523-11321-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127579900523-11321-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.14.67:0
--local_ip_for_outbound_sockets=127.11.14.67
--webserver_interface=127.11.14.67
--webserver_port=0
--tserver_master_addrs=127.11.14.126:38513
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.67.keytab
--principal=kudu/127.11.14.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.14.84:44599
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127579900523-11321-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127579900523-11321-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127579900523-11321-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:37:17.209887 31408 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:37:17.210139 31408 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:37:17.210219 31408 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:37:17.214293 31408 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:37:17.214350 31408 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:37:17.214458 31408 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.14.67
I20260807 18:37:17.219745 31408 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.14.84:44599
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127579900523-11321-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127579900523-11321-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.67.keytab
--principal=kudu/127.11.14.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.14.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127579900523-11321-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.14.67
--webserver_port=0
--tserver_master_addrs=127.11.14.126:38513
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31408
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.14.67
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127579900523-11321-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:37:17.220822 31408 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:37:17.221853 31408 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:37:17.231592 31413 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:37:17.231597 31416 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:37:17.231630 31414 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:37:17.231817 31408 server_base.cc:1034] running on GCE node
I20260807 18:37:17.232314 31408 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:37:17.232996 31408 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:37:17.234185 31408 hybrid_clock.cc:648] HybridClock initialized: now 1786127837234162 us; error 35 us; skew 500 ppm
Aug 07 18:37:17 dist-test-slave-6k22 krb5kdc[30992](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127837, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:37:17.237236 31408 init.cc:377] Logged in from keytab as kudu/127.11.14.67@KRBTEST.COM (short username kudu)
I20260807 18:37:17.238085 31408 webserver.cc:460] Webserver started at http://127.11.14.67:40033/ using document root <none> and password file <none>
I20260807 18:37:17.238678 31408 fs_manager.cc:359] Metadata directory not provided
I20260807 18:37:17.238771 31408 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:37:17.239022 31408 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:37:17.241122 31408 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127579900523-11321-0/minicluster-data/ts-2/data/instance:
uuid: "83bef5c3d15e438c8f2c16f9a8130043"
format_stamp: "Formatted at 2026-08-07 18:37:17 on dist-test-slave-6k22"
server_key: "f911112013051096b79b3c6a501d3e72"
server_key_iv: "eeef1fbcf769592e529e97b187611212"
server_key_version: "encryptionkey@0"
I20260807 18:37:17.241696 31408 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127579900523-11321-0/minicluster-data/ts-2/wal/instance:
uuid: "83bef5c3d15e438c8f2c16f9a8130043"
format_stamp: "Formatted at 2026-08-07 18:37:17 on dist-test-slave-6k22"
server_key: "f911112013051096b79b3c6a501d3e72"
server_key_iv: "eeef1fbcf769592e529e97b187611212"
server_key_version: "encryptionkey@0"
I20260807 18:37:17.245411 31408 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.001s
I20260807 18:37:17.247977 31423 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:37:17.249187 31408 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 18:37:17.249323 31408 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127579900523-11321-0/minicluster-data/ts-2/data,/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127579900523-11321-0/minicluster-data/ts-2/wal
uuid: "83bef5c3d15e438c8f2c16f9a8130043"
format_stamp: "Formatted at 2026-08-07 18:37:17 on dist-test-slave-6k22"
server_key: "f911112013051096b79b3c6a501d3e72"
server_key_iv: "eeef1fbcf769592e529e97b187611212"
server_key_version: "encryptionkey@0"
I20260807 18:37:17.249432 31408 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127579900523-11321-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127579900523-11321-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127579900523-11321-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:37:17.269326 31408 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:37:17.272641 31408 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:37:17.272855 31408 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:37:17.273474 31408 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:37:17.274330 31408 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:37:17.274401 31408 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:37:17.274492 31408 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:37:17.274541 31408 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:37:17.283440 31408 rpc_server.cc:307] RPC server started. Bound to: 127.11.14.67:42591
I20260807 18:37:17.283511 31548 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.14.67:42591 every 8 connection(s)
I20260807 18:37:17.284405 31408 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127579900523-11321-0/minicluster-data/ts-2/data/info.pb
I20260807 18:37:17.291590 11321 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkz56XN/build/debug/bin/kudu as pid 31408
I20260807 18:37:17.291754 11321 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127579900523-11321-0/minicluster-data/ts-2/wal/instance
I20260807 18:37:17.292049 11321 external_mini_cluster.cc:1428] Setting key d33b3b0a392f3abc9db116407a371458
Aug 07 18:37:17 dist-test-slave-6k22 krb5kdc[30992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127837, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.67@KRBTEST.COM for kudu/127.11.14.126@KRBTEST.COM
I20260807 18:37:17.298230 31086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:17.286235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.67:57061 (local address 127.11.14.126:38513)
0807 18:37:17.286374 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0807 18:37:17.286378 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:37:17.287238 (+ 860us) server_negotiation.cc:407] Connection header received
0807 18:37:17.288845 (+ 1607us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:37:17.288848 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:37:17.288915 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:37:17.289005 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:37:17.290197 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:17.290730 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:17.292031 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:17.292304 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:17.292416 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:17.295169 (+ 2753us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:37:17.295172 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:37:17.295174 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:37:17.295200 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:37:17.296509 (+ 1309us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:37:17.296970 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:37:17.296972 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:37:17.296973 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:37:17.297045 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:37:17.297329 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:37:17.297331 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:37:17.297332 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:37:17.297497 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:37:17.297573 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0807 18:37:17.298021 (+ 448us) server_negotiation.cc:299] Negotiation successful
0807 18:37:17.298115 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260807 18:37:17.299048 31551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:17.286630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:38513 (local address 127.11.14.67:57061)
0807 18:37:17.287074 (+ 444us) negotiation.cc:107] Waiting for socket to connect
0807 18:37:17.287148 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0807 18:37:17.288612 (+ 1464us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:37:17.289191 (+ 579us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:37:17.289197 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:37:17.289565 (+ 368us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:37:17.290034 (+ 469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:37:17.290048 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:37:17.290867 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:37:17.290870 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:37:17.291912 (+ 1042us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:37:17.291922 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:37:17.292457 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:37:17.292461 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:37:17.292642 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:17.293285 (+ 643us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:37:17.293307 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:37:17.295022 (+ 1715us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:37:17.296613 (+ 1591us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:37:17.296618 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:37:17.296629 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:37:17.296884 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:37:17.297123 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:37:17.297125 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:37:17.297126 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:37:17.297251 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:37:17.297585 (+ 334us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:37:17.297590 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:37:17.297820 (+ 230us) client_negotiation.cc:769] Sending connection context
0807 18:37:17.297974 (+ 154us) client_negotiation.cc:240] Negotiation successful
0807 18:37:17.298212 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":120,"threads_started":1}
I20260807 18:37:17.300304 31549 heartbeater.cc:347] Connected to a master server at 127.11.14.126:38513
I20260807 18:37:17.300509 31549 heartbeater.cc:464] Registering TS with master...
I20260807 18:37:17.300940 31549 heartbeater.cc:511] Master 127.11.14.126:38513 requested a full tablet report, sending...
I20260807 18:37:17.301937 31045 ts_manager.cc:194] Registered new tserver with Master: 83bef5c3d15e438c8f2c16f9a8130043 (127.11.14.67:42591)
I20260807 18:37:17.302448 31045 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.14.67@KRBTEST.COM'} at 127.11.14.67:57061
I20260807 18:37:17.305960 11321 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 18:37:17.315279 31086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:17.307918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48382 (local address 127.11.14.126:38513)
0807 18:37:17.308040 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0807 18:37:17.308044 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:37:17.308262 (+ 218us) server_negotiation.cc:407] Connection header received
0807 18:37:17.308375 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:37:17.308377 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:37:17.308413 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:37:17.308478 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:37:17.309236 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:17.309722 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:17.310553 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:17.310755 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:17.310834 (+ 79us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:17.312352 (+ 1518us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:37:17.312356 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:37:17.312357 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:37:17.312379 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:37:17.313783 (+ 1404us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:37:17.314217 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:37:17.314219 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:37:17.314221 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:37:17.314288 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:37:17.314574 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:37:17.314576 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:37:17.314577 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:37:17.314763 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:37:17.314878 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0807 18:37:17.315091 (+ 213us) server_negotiation.cc:299] Negotiation successful
0807 18:37:17.315164 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260807 18:37:17.318197 31045 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48382:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 18:37:17.321408 31045 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 18:37:17.336683 31566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:17.330866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.67:42591 (local address 127.0.0.1:57964)
0807 18:37:17.331668 (+ 802us) negotiation.cc:107] Waiting for socket to connect
0807 18:37:17.331682 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 18:37:17.331760 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:37:17.332510 (+ 750us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:37:17.332513 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:37:17.332532 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:37:17.332615 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:37:17.332621 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:37:17.333608 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:37:17.333614 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:37:17.335130 (+ 1516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:37:17.335142 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:37:17.336148 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:37:17.336152 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:37:17.336254 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:17.336292 (+ 38us) client_negotiation.cc:769] Sending connection context
0807 18:37:17.336405 (+ 113us) client_negotiation.cc:240] Negotiation successful
0807 18:37:17.336524 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":742,"mutex_wait_us":40,"thread_start_us":89,"threads_started":1}
I20260807 18:37:17.336833 31567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:17.330905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33254 (local address 127.11.14.66:38239)
0807 18:37:17.331657 (+ 752us) server_negotiation.cc:206] Beginning negotiation
0807 18:37:17.331676 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0807 18:37:17.331701 (+ 25us) server_negotiation.cc:407] Connection header received
0807 18:37:17.331790 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:37:17.331793 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:37:17.331942 (+ 149us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:37:17.332264 (+ 322us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:37:17.332438 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:17.333635 (+ 1197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:17.335365 (+ 1730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:17.336295 (+ 930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:17.336404 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:17.336544 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0807 18:37:17.336600 (+ 56us) server_negotiation.cc:299] Negotiation successful
0807 18:37:17.336681 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":619,"thread_start_us":155,"threads_started":1}
I20260807 18:37:17.337210 31562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:17.330070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.65:40479 (local address 127.0.0.1:55614)
0807 18:37:17.331207 (+ 1137us) negotiation.cc:107] Waiting for socket to connect
0807 18:37:17.331262 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0807 18:37:17.331403 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:37:17.332038 (+ 635us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:37:17.332047 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:37:17.332075 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:37:17.332264 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:37:17.332273 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:37:17.333778 (+ 1505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:37:17.333781 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:37:17.335191 (+ 1410us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:37:17.335202 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:37:17.336809 (+ 1607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:37:17.336814 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:37:17.336915 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:17.336941 (+ 26us) client_negotiation.cc:769] Sending connection context
0807 18:37:17.337012 (+ 71us) client_negotiation.cc:240] Negotiation successful
0807 18:37:17.337077 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":202,"mutex_wait_us":100,"spinlock_wait_cycles":3712,"thread_start_us":98,"threads_started":1}
I20260807 18:37:17.336731 31563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:17.330150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.66:38239 (local address 127.0.0.1:33254)
0807 18:37:17.331207 (+ 1057us) negotiation.cc:107] Waiting for socket to connect
0807 18:37:17.331262 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0807 18:37:17.331403 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:37:17.332111 (+ 708us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:37:17.332114 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:37:17.332126 (+ 12us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:37:17.332325 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:37:17.332336 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:37:17.333786 (+ 1450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:37:17.333790 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:37:17.335238 (+ 1448us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:37:17.335252 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:37:17.336402 (+ 1150us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:37:17.336406 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:37:17.336484 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:17.336505 (+ 21us) client_negotiation.cc:769] Sending connection context
0807 18:37:17.336563 (+ 58us) client_negotiation.cc:240] Negotiation successful
0807 18:37:17.336620 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":585,"mutex_wait_us":105,"spinlock_wait_cycles":899200,"thread_start_us":90,"threads_started":1}
I20260807 18:37:17.337428 31565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:17.330211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55614 (local address 127.11.14.65:40479)
0807 18:37:17.331100 (+ 889us) server_negotiation.cc:206] Beginning negotiation
0807 18:37:17.331116 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 18:37:17.331359 (+ 243us) server_negotiation.cc:407] Connection header received
0807 18:37:17.331627 (+ 268us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:37:17.331634 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:37:17.331867 (+ 233us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:37:17.332027 (+ 160us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:37:17.332410 (+ 383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:17.333617 (+ 1207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:17.335353 (+ 1736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:17.336290 (+ 937us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:17.336404 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:17.336545 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0807 18:37:17.337093 (+ 548us) server_negotiation.cc:299] Negotiation successful
0807 18:37:17.337203 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":769,"thread_start_us":110,"threads_started":1}
I20260807 18:37:17.337812 31568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:17.331059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57964 (local address 127.11.14.67:42591)
0807 18:37:17.332018 (+ 959us) server_negotiation.cc:206] Beginning negotiation
0807 18:37:17.332031 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 18:37:17.332051 (+ 20us) server_negotiation.cc:407] Connection header received
0807 18:37:17.332126 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:37:17.332131 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:37:17.332336 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:37:17.332520 (+ 184us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:37:17.332734 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:17.333477 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:17.335264 (+ 1787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:17.336042 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:17.336150 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:17.336313 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0807 18:37:17.337546 (+ 1233us) server_negotiation.cc:299] Negotiation successful
0807 18:37:17.337643 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":853,"thread_start_us":117,"threads_started":1}
I20260807 18:37:17.340415 31181 tablet_service.cc:1467] Processing CreateTablet for tablet c0c61659859f43b893830288e60368ac (DEFAULT_TABLE table=test-table [id=6ce92412fa364cb19b478076e611545f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:37:17.340641 31331 tablet_service.cc:1467] Processing CreateTablet for tablet c0c61659859f43b893830288e60368ac (DEFAULT_TABLE table=test-table [id=6ce92412fa364cb19b478076e611545f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:37:17.340595 31475 tablet_service.cc:1467] Processing CreateTablet for tablet c0c61659859f43b893830288e60368ac (DEFAULT_TABLE table=test-table [id=6ce92412fa364cb19b478076e611545f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:37:17.341565 31181 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c0c61659859f43b893830288e60368ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:37:17.341617 31331 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c0c61659859f43b893830288e60368ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:37:17.341622 31475 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c0c61659859f43b893830288e60368ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:37:17.347978 31569 tablet_bootstrap.cc:492] T c0c61659859f43b893830288e60368ac P 39cfa69c502e42b295bb8d2c4b984f70: Bootstrap starting.
I20260807 18:37:17.349064 31571 tablet_bootstrap.cc:492] T c0c61659859f43b893830288e60368ac P 83bef5c3d15e438c8f2c16f9a8130043: Bootstrap starting.
I20260807 18:37:17.349480 31570 tablet_bootstrap.cc:492] T c0c61659859f43b893830288e60368ac P 5618b1865a3641509fd0aef10fa3790e: Bootstrap starting.
I20260807 18:37:17.350219 31569 tablet_bootstrap.cc:654] T c0c61659859f43b893830288e60368ac P 39cfa69c502e42b295bb8d2c4b984f70: Neither blocks nor log segments found. Creating new log.
I20260807 18:37:17.350956 31569 log.cc:824] T c0c61659859f43b893830288e60368ac P 39cfa69c502e42b295bb8d2c4b984f70: Log is configured to *not* fsync() on all Append() calls
I20260807 18:37:17.351740 31570 tablet_bootstrap.cc:654] T c0c61659859f43b893830288e60368ac P 5618b1865a3641509fd0aef10fa3790e: Neither blocks nor log segments found. Creating new log.
I20260807 18:37:17.352716 31570 log.cc:824] T c0c61659859f43b893830288e60368ac P 5618b1865a3641509fd0aef10fa3790e: Log is configured to *not* fsync() on all Append() calls
I20260807 18:37:17.353235 31569 tablet_bootstrap.cc:492] T c0c61659859f43b893830288e60368ac P 39cfa69c502e42b295bb8d2c4b984f70: No bootstrap required, opened a new log
I20260807 18:37:17.353404 31571 tablet_bootstrap.cc:654] T c0c61659859f43b893830288e60368ac P 83bef5c3d15e438c8f2c16f9a8130043: Neither blocks nor log segments found. Creating new log.
I20260807 18:37:17.353456 31569 ts_tablet_manager.cc:1397] T c0c61659859f43b893830288e60368ac P 39cfa69c502e42b295bb8d2c4b984f70: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260807 18:37:17.354439 31571 log.cc:824] T c0c61659859f43b893830288e60368ac P 83bef5c3d15e438c8f2c16f9a8130043: Log is configured to *not* fsync() on all Append() calls
I20260807 18:37:17.354722 31570 tablet_bootstrap.cc:492] T c0c61659859f43b893830288e60368ac P 5618b1865a3641509fd0aef10fa3790e: No bootstrap required, opened a new log
I20260807 18:37:17.354934 31570 ts_tablet_manager.cc:1397] T c0c61659859f43b893830288e60368ac P 5618b1865a3641509fd0aef10fa3790e: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260807 18:37:17.356163 31571 tablet_bootstrap.cc:492] T c0c61659859f43b893830288e60368ac P 83bef5c3d15e438c8f2c16f9a8130043: No bootstrap required, opened a new log
I20260807 18:37:17.356362 31571 ts_tablet_manager.cc:1397] T c0c61659859f43b893830288e60368ac P 83bef5c3d15e438c8f2c16f9a8130043: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260807 18:37:17.357895 31569 raft_consensus.cc:348] T c0c61659859f43b893830288e60368ac P 39cfa69c502e42b295bb8d2c4b984f70 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39cfa69c502e42b295bb8d2c4b984f70" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 40479 } } peers { permanent_uuid: "5618b1865a3641509fd0aef10fa3790e" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 38239 } } peers { permanent_uuid: "83bef5c3d15e438c8f2c16f9a8130043" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 42591 } }
I20260807 18:37:17.358170 31569 raft_consensus.cc:374] T c0c61659859f43b893830288e60368ac P 39cfa69c502e42b295bb8d2c4b984f70 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:37:17.358256 31569 raft_consensus.cc:729] T c0c61659859f43b893830288e60368ac P 39cfa69c502e42b295bb8d2c4b984f70 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39cfa69c502e42b295bb8d2c4b984f70, State: Initialized, Role: FOLLOWER
I20260807 18:37:17.358790 31569 consensus_queue.cc:260] T c0c61659859f43b893830288e60368ac P 39cfa69c502e42b295bb8d2c4b984f70 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39cfa69c502e42b295bb8d2c4b984f70" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 40479 } } peers { permanent_uuid: "5618b1865a3641509fd0aef10fa3790e" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 38239 } } peers { permanent_uuid: "83bef5c3d15e438c8f2c16f9a8130043" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 42591 } }
I20260807 18:37:17.358723 31570 raft_consensus.cc:348] T c0c61659859f43b893830288e60368ac P 5618b1865a3641509fd0aef10fa3790e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39cfa69c502e42b295bb8d2c4b984f70" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 40479 } } peers { permanent_uuid: "5618b1865a3641509fd0aef10fa3790e" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 38239 } } peers { permanent_uuid: "83bef5c3d15e438c8f2c16f9a8130043" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 42591 } }
I20260807 18:37:17.358983 31570 raft_consensus.cc:374] T c0c61659859f43b893830288e60368ac P 5618b1865a3641509fd0aef10fa3790e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:37:17.359038 31570 raft_consensus.cc:729] T c0c61659859f43b893830288e60368ac P 5618b1865a3641509fd0aef10fa3790e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5618b1865a3641509fd0aef10fa3790e, State: Initialized, Role: FOLLOWER
I20260807 18:37:17.359468 31570 consensus_queue.cc:260] T c0c61659859f43b893830288e60368ac P 5618b1865a3641509fd0aef10fa3790e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39cfa69c502e42b295bb8d2c4b984f70" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 40479 } } peers { permanent_uuid: "5618b1865a3641509fd0aef10fa3790e" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 38239 } } peers { permanent_uuid: "83bef5c3d15e438c8f2c16f9a8130043" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 42591 } }
I20260807 18:37:17.359447 31571 raft_consensus.cc:348] T c0c61659859f43b893830288e60368ac P 83bef5c3d15e438c8f2c16f9a8130043 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39cfa69c502e42b295bb8d2c4b984f70" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 40479 } } peers { permanent_uuid: "5618b1865a3641509fd0aef10fa3790e" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 38239 } } peers { permanent_uuid: "83bef5c3d15e438c8f2c16f9a8130043" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 42591 } }
I20260807 18:37:17.359761 31571 raft_consensus.cc:374] T c0c61659859f43b893830288e60368ac P 83bef5c3d15e438c8f2c16f9a8130043 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:37:17.359824 31571 raft_consensus.cc:729] T c0c61659859f43b893830288e60368ac P 83bef5c3d15e438c8f2c16f9a8130043 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83bef5c3d15e438c8f2c16f9a8130043, State: Initialized, Role: FOLLOWER
I20260807 18:37:17.359920 31569 ts_tablet_manager.cc:1428] T c0c61659859f43b893830288e60368ac P 39cfa69c502e42b295bb8d2c4b984f70: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260807 18:37:17.360301 31255 heartbeater.cc:503] Master 127.11.14.126:38513 was elected leader, sending a full tablet report...
I20260807 18:37:17.360246 31571 consensus_queue.cc:260] T c0c61659859f43b893830288e60368ac P 83bef5c3d15e438c8f2c16f9a8130043 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39cfa69c502e42b295bb8d2c4b984f70" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 40479 } } peers { permanent_uuid: "5618b1865a3641509fd0aef10fa3790e" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 38239 } } peers { permanent_uuid: "83bef5c3d15e438c8f2c16f9a8130043" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 42591 } }
I20260807 18:37:17.360427 31570 ts_tablet_manager.cc:1428] T c0c61659859f43b893830288e60368ac P 5618b1865a3641509fd0aef10fa3790e: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260807 18:37:17.360731 31400 heartbeater.cc:503] Master 127.11.14.126:38513 was elected leader, sending a full tablet report...
I20260807 18:37:17.361171 31571 ts_tablet_manager.cc:1428] T c0c61659859f43b893830288e60368ac P 83bef5c3d15e438c8f2c16f9a8130043: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260807 18:37:17.361483 31549 heartbeater.cc:503] Master 127.11.14.126:38513 was elected leader, sending a full tablet report...
I20260807 18:37:17.473443 31575 raft_consensus.cc:482] T c0c61659859f43b893830288e60368ac P 39cfa69c502e42b295bb8d2c4b984f70 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:37:17.473726 31575 raft_consensus.cc:504] T c0c61659859f43b893830288e60368ac P 39cfa69c502e42b295bb8d2c4b984f70 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39cfa69c502e42b295bb8d2c4b984f70" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 40479 } } peers { permanent_uuid: "5618b1865a3641509fd0aef10fa3790e" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 38239 } } peers { permanent_uuid: "83bef5c3d15e438c8f2c16f9a8130043" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 42591 } }
I20260807 18:37:17.475076 31575 leader_election.cc:288] T c0c61659859f43b893830288e60368ac P 39cfa69c502e42b295bb8d2c4b984f70 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5618b1865a3641509fd0aef10fa3790e (127.11.14.66:38239), 83bef5c3d15e438c8f2c16f9a8130043 (127.11.14.67:42591)
I20260807 18:37:17.479328 31568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:17.475626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.65:53197 (local address 127.11.14.67:42591)
0807 18:37:17.475759 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0807 18:37:17.475764 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:37:17.476027 (+ 263us) server_negotiation.cc:407] Connection header received
0807 18:37:17.476125 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:37:17.476127 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:37:17.476196 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:37:17.476277 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:37:17.476603 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:17.477089 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:17.478301 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:17.478729 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:17.478859 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:17.478916 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0807 18:37:17.479125 (+ 209us) server_negotiation.cc:299] Negotiation successful
0807 18:37:17.479199 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260807 18:37:17.479346 31579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:17.475459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.67:42591 (local address 127.11.14.65:53197)
0807 18:37:17.475891 (+ 432us) negotiation.cc:107] Waiting for socket to connect
0807 18:37:17.475907 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 18:37:17.476039 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:37:17.476312 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:37:17.476315 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:37:17.476335 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:37:17.476463 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:37:17.476468 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:37:17.477211 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:37:17.477214 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:37:17.478165 (+ 951us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:37:17.478175 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:37:17.478880 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:37:17.478884 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:37:17.478984 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:17.479013 (+ 29us) client_negotiation.cc:769] Sending connection context
0807 18:37:17.479076 (+ 63us) client_negotiation.cc:240] Negotiation successful
0807 18:37:17.479164 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":350,"mutex_wait_us":7,"thread_start_us":139,"threads_started":1}
I20260807 18:37:17.479377 31567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:17.475474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.65:55225 (local address 127.11.14.66:38239)
0807 18:37:17.475663 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0807 18:37:17.475669 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:37:17.475892 (+ 223us) server_negotiation.cc:407] Connection header received
0807 18:37:17.476034 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:37:17.476038 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:37:17.476097 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:37:17.476180 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:37:17.476558 (+ 378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:17.477087 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:17.478333 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:17.478761 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:17.478902 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:17.478957 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0807 18:37:17.479180 (+ 223us) server_negotiation.cc:299] Negotiation successful
0807 18:37:17.479264 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260807 18:37:17.479367 31578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:17.475318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.66:38239 (local address 127.11.14.65:55225)
0807 18:37:17.475770 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0807 18:37:17.475787 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 18:37:17.475924 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:37:17.476204 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:37:17.476207 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:37:17.476226 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:37:17.476437 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:37:17.476445 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:37:17.477206 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:37:17.477211 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:37:17.478169 (+ 958us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:37:17.478180 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:37:17.478909 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:37:17.478913 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:37:17.479028 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:17.479046 (+ 18us) client_negotiation.cc:769] Sending connection context
0807 18:37:17.479147 (+ 101us) client_negotiation.cc:240] Negotiation successful
0807 18:37:17.479214 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":386,"mutex_wait_us":89,"thread_start_us":140,"threads_started":1}
I20260807 18:37:17.480172 31498 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c0c61659859f43b893830288e60368ac" candidate_uuid: "39cfa69c502e42b295bb8d2c4b984f70" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83bef5c3d15e438c8f2c16f9a8130043" is_pre_election: true
I20260807 18:37:17.480329 31353 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c0c61659859f43b893830288e60368ac" candidate_uuid: "39cfa69c502e42b295bb8d2c4b984f70" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5618b1865a3641509fd0aef10fa3790e" is_pre_election: true
I20260807 18:37:17.480559 31498 raft_consensus.cc:2454] T c0c61659859f43b893830288e60368ac P 83bef5c3d15e438c8f2c16f9a8130043 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39cfa69c502e42b295bb8d2c4b984f70 in term 0.
I20260807 18:37:17.480648 31353 raft_consensus.cc:2454] T c0c61659859f43b893830288e60368ac P 5618b1865a3641509fd0aef10fa3790e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39cfa69c502e42b295bb8d2c4b984f70 in term 0.
I20260807 18:37:17.481149 31128 leader_election.cc:302] T c0c61659859f43b893830288e60368ac P 39cfa69c502e42b295bb8d2c4b984f70 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 39cfa69c502e42b295bb8d2c4b984f70, 5618b1865a3641509fd0aef10fa3790e, 83bef5c3d15e438c8f2c16f9a8130043; no voters:
I20260807 18:37:17.481468 31575 raft_consensus.cc:2781] T c0c61659859f43b893830288e60368ac P 39cfa69c502e42b295bb8d2c4b984f70 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:37:17.481570 31575 raft_consensus.cc:482] T c0c61659859f43b893830288e60368ac P 39cfa69c502e42b295bb8d2c4b984f70 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:37:17.481628 31575 raft_consensus.cc:3037] T c0c61659859f43b893830288e60368ac P 39cfa69c502e42b295bb8d2c4b984f70 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:37:17.482702 31575 raft_consensus.cc:504] T c0c61659859f43b893830288e60368ac P 39cfa69c502e42b295bb8d2c4b984f70 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39cfa69c502e42b295bb8d2c4b984f70" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 40479 } } peers { permanent_uuid: "5618b1865a3641509fd0aef10fa3790e" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 38239 } } peers { permanent_uuid: "83bef5c3d15e438c8f2c16f9a8130043" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 42591 } }
I20260807 18:37:17.483108 31575 leader_election.cc:288] T c0c61659859f43b893830288e60368ac P 39cfa69c502e42b295bb8d2c4b984f70 [CANDIDATE]: Term 1 election: Requested vote from peers 5618b1865a3641509fd0aef10fa3790e (127.11.14.66:38239), 83bef5c3d15e438c8f2c16f9a8130043 (127.11.14.67:42591)
I20260807 18:37:17.483453 31353 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c0c61659859f43b893830288e60368ac" candidate_uuid: "39cfa69c502e42b295bb8d2c4b984f70" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5618b1865a3641509fd0aef10fa3790e"
I20260807 18:37:17.483507 31498 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c0c61659859f43b893830288e60368ac" candidate_uuid: "39cfa69c502e42b295bb8d2c4b984f70" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83bef5c3d15e438c8f2c16f9a8130043"
I20260807 18:37:17.483608 31353 raft_consensus.cc:3037] T c0c61659859f43b893830288e60368ac P 5618b1865a3641509fd0aef10fa3790e [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:37:17.483672 31498 raft_consensus.cc:3037] T c0c61659859f43b893830288e60368ac P 83bef5c3d15e438c8f2c16f9a8130043 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:37:17.484709 31498 raft_consensus.cc:2454] T c0c61659859f43b893830288e60368ac P 83bef5c3d15e438c8f2c16f9a8130043 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39cfa69c502e42b295bb8d2c4b984f70 in term 1.
I20260807 18:37:17.484766 31353 raft_consensus.cc:2454] T c0c61659859f43b893830288e60368ac P 5618b1865a3641509fd0aef10fa3790e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39cfa69c502e42b295bb8d2c4b984f70 in term 1.
I20260807 18:37:17.485028 31128 leader_election.cc:302] T c0c61659859f43b893830288e60368ac P 39cfa69c502e42b295bb8d2c4b984f70 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39cfa69c502e42b295bb8d2c4b984f70, 83bef5c3d15e438c8f2c16f9a8130043; no voters:
I20260807 18:37:17.485213 31575 raft_consensus.cc:2781] T c0c61659859f43b893830288e60368ac P 39cfa69c502e42b295bb8d2c4b984f70 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:37:17.485441 31575 raft_consensus.cc:686] T c0c61659859f43b893830288e60368ac P 39cfa69c502e42b295bb8d2c4b984f70 [term 1 LEADER]: Becoming Leader. State: Replica: 39cfa69c502e42b295bb8d2c4b984f70, State: Running, Role: LEADER
I20260807 18:37:17.485833 31575 consensus_queue.cc:237] T c0c61659859f43b893830288e60368ac P 39cfa69c502e42b295bb8d2c4b984f70 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39cfa69c502e42b295bb8d2c4b984f70" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 40479 } } peers { permanent_uuid: "5618b1865a3641509fd0aef10fa3790e" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 38239 } } peers { permanent_uuid: "83bef5c3d15e438c8f2c16f9a8130043" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 42591 } }
I20260807 18:37:17.489182 31045 catalog_manager.cc:5495] T c0c61659859f43b893830288e60368ac P 39cfa69c502e42b295bb8d2c4b984f70 reported cstate change: term changed from 0 to 1, leader changed from <none> to 39cfa69c502e42b295bb8d2c4b984f70 (127.11.14.65). New cstate: current_term: 1 leader_uuid: "39cfa69c502e42b295bb8d2c4b984f70" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39cfa69c502e42b295bb8d2c4b984f70" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 40479 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5618b1865a3641509fd0aef10fa3790e" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 38239 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "83bef5c3d15e438c8f2c16f9a8130043" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 42591 } health_report { overall_health: UNKNOWN } } }
I20260807 18:37:17.526216 31565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:17.522428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55618 (local address 127.11.14.65:40479)
0807 18:37:17.522585 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0807 18:37:17.522589 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:37:17.522604 (+ 15us) server_negotiation.cc:407] Connection header received
0807 18:37:17.522766 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:37:17.522769 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:37:17.522837 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:37:17.522953 (+ 116us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:37:17.523227 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:17.523810 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:17.524693 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:17.524913 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:17.525020 (+ 107us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:17.525331 (+ 311us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:37:17.525733 (+ 402us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:37:17.525908 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0807 18:37:17.525992 (+ 84us) server_negotiation.cc:299] Negotiation successful
0807 18:37:17.526033 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260807 18:37:17.532599 31353 raft_consensus.cc:1261] T c0c61659859f43b893830288e60368ac P 5618b1865a3641509fd0aef10fa3790e [term 1 FOLLOWER]: Refusing update from remote peer 39cfa69c502e42b295bb8d2c4b984f70: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:37:17.532611 31498 raft_consensus.cc:1261] T c0c61659859f43b893830288e60368ac P 83bef5c3d15e438c8f2c16f9a8130043 [term 1 FOLLOWER]: Refusing update from remote peer 39cfa69c502e42b295bb8d2c4b984f70: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:37:17.533366 31580 consensus_queue.cc:1035] T c0c61659859f43b893830288e60368ac P 39cfa69c502e42b295bb8d2c4b984f70 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5618b1865a3641509fd0aef10fa3790e" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 38239 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:37:17.533595 31575 consensus_queue.cc:1035] T c0c61659859f43b893830288e60368ac P 39cfa69c502e42b295bb8d2c4b984f70 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83bef5c3d15e438c8f2c16f9a8130043" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 42591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260807 18:37:17.535341 31550 tablet.cc:2369] T c0c61659859f43b893830288e60368ac P 83bef5c3d15e438c8f2c16f9a8130043: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:37:17.542788 31585 mvcc.cc:205] Tried to move back new op lower bound from 7315979622525562880 to 7315979622344716288. Current Snapshot: MvccSnapshot[applied={T|T < 7315979622525562880}]
I20260807 18:37:17.550972 31043 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48382:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:37:17.551132 31043 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48382:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:37:17.553730 31043 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b03547278cc84705bb7578776cb192d5: Sending DeleteTablet for 3 replicas of tablet c0c61659859f43b893830288e60368ac
I20260807 18:37:17.554524 31331 tablet_service.cc:1514] Processing DeleteTablet for tablet c0c61659859f43b893830288e60368ac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:37:17 UTC) from {username='kudu', principal='kudu/127.11.14.126@KRBTEST.COM'} at 127.0.0.1:33254
I20260807 18:37:17.554524 31181 tablet_service.cc:1514] Processing DeleteTablet for tablet c0c61659859f43b893830288e60368ac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:37:17 UTC) from {username='kudu', principal='kudu/127.11.14.126@KRBTEST.COM'} at 127.0.0.1:55614
I20260807 18:37:17.555060 31595 tablet_replica.cc:330] T c0c61659859f43b893830288e60368ac P 39cfa69c502e42b295bb8d2c4b984f70: stopping tablet replica
I20260807 18:37:17.555059 31475 tablet_service.cc:1514] Processing DeleteTablet for tablet c0c61659859f43b893830288e60368ac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:37:17 UTC) from {username='kudu', principal='kudu/127.11.14.126@KRBTEST.COM'} at 127.0.0.1:57964
I20260807 18:37:17.555397 31595 raft_consensus.cc:2229] T c0c61659859f43b893830288e60368ac P 39cfa69c502e42b295bb8d2c4b984f70 [term 1 LEADER]: Raft consensus shutting down.
I20260807 18:37:17.555722 31596 tablet_replica.cc:330] T c0c61659859f43b893830288e60368ac P 83bef5c3d15e438c8f2c16f9a8130043: stopping tablet replica
I20260807 18:37:17.555824 11321 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkz56XN/build/debug/bin/kudu with pid 31111
I20260807 18:37:17.555842 31595 raft_consensus.cc:2258] T c0c61659859f43b893830288e60368ac P 39cfa69c502e42b295bb8d2c4b984f70 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:37:17.556200 31594 tablet_replica.cc:330] T c0c61659859f43b893830288e60368ac P 5618b1865a3641509fd0aef10fa3790e: stopping tablet replica
I20260807 18:37:17.556478 31594 raft_consensus.cc:2229] T c0c61659859f43b893830288e60368ac P 5618b1865a3641509fd0aef10fa3790e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:37:17.556474 31596 raft_consensus.cc:2229] T c0c61659859f43b893830288e60368ac P 83bef5c3d15e438c8f2c16f9a8130043 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:37:17.556787 31594 raft_consensus.cc:2258] T c0c61659859f43b893830288e60368ac P 5618b1865a3641509fd0aef10fa3790e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:37:17.556825 31596 raft_consensus.cc:2258] T c0c61659859f43b893830288e60368ac P 83bef5c3d15e438c8f2c16f9a8130043 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:37:17.559512 31594 ts_tablet_manager.cc:1905] T c0c61659859f43b893830288e60368ac P 5618b1865a3641509fd0aef10fa3790e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:37:17.560386 31596 ts_tablet_manager.cc:1905] T c0c61659859f43b893830288e60368ac P 83bef5c3d15e438c8f2c16f9a8130043: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:37:17.561924 31594 ts_tablet_manager.cc:1918] T c0c61659859f43b893830288e60368ac P 5618b1865a3641509fd0aef10fa3790e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:37:17.562047 31594 log.cc:1196] T c0c61659859f43b893830288e60368ac P 5618b1865a3641509fd0aef10fa3790e: Deleting WAL directory at /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127579900523-11321-0/minicluster-data/ts-1/wal/wals/c0c61659859f43b893830288e60368ac
I20260807 18:37:17.562347 31594 ts_tablet_manager.cc:1939] T c0c61659859f43b893830288e60368ac P 5618b1865a3641509fd0aef10fa3790e: Deleting consensus metadata
I20260807 18:37:17.563550 31027 catalog_manager.cc:4841] TS 5618b1865a3641509fd0aef10fa3790e (127.11.14.66:38239): tablet c0c61659859f43b893830288e60368ac (table test-table [id=6ce92412fa364cb19b478076e611545f]) successfully deleted
I20260807 18:37:17.563827 31596 ts_tablet_manager.cc:1918] T c0c61659859f43b893830288e60368ac P 83bef5c3d15e438c8f2c16f9a8130043: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:37:17.563931 31596 log.cc:1196] T c0c61659859f43b893830288e60368ac P 83bef5c3d15e438c8f2c16f9a8130043: Deleting WAL directory at /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786127579900523-11321-0/minicluster-data/ts-2/wal/wals/c0c61659859f43b893830288e60368ac
I20260807 18:37:17.564309 31596 ts_tablet_manager.cc:1939] T c0c61659859f43b893830288e60368ac P 83bef5c3d15e438c8f2c16f9a8130043: Deleting consensus metadata
I20260807 18:37:17.565605 31026 catalog_manager.cc:4841] TS 83bef5c3d15e438c8f2c16f9a8130043 (127.11.14.67:42591): tablet c0c61659859f43b893830288e60368ac (table test-table [id=6ce92412fa364cb19b478076e611545f]) successfully deleted
W20260807 18:37:17.567247 31029 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.14.65:40479 (error 108)
I20260807 18:37:17.568130 11321 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkz56XN/build/debug/bin/kudu with pid 31260
I20260807 18:37:17.568403 31562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:17.568104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.65:40479 (local address 127.0.0.1:55630)
0807 18:37:17.568240 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0807 18:37:17.568308 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.65:40479: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260807 18:37:17.570047 31029 catalog_manager.cc:4584] TS 39cfa69c502e42b295bb8d2c4b984f70 (127.11.14.65:40479): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c0c61659859f43b893830288e60368ac: Network error: Client connection negotiation failed: client connection to 127.11.14.65:40479: connect: Connection refused (error 111)
I20260807 18:37:17.579042 11321 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkz56XN/build/debug/bin/kudu with pid 31408
I20260807 18:37:17.587819 11321 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkz56XN/build/debug/bin/kudu with pid 31006
2026-08-07T18:37:17Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2490 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-07T18:37:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:37:17Z Disabled control of system clock
I20260807 18:37:17.622552 11321 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkz56XN/build/debug/bin/kudu
/tmp/dist-test-taskkz56XN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786127579900523-11321-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786127579900523-11321-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786127579900523-11321-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786127579900523-11321-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.14.126:33069
--webserver_interface=127.11.14.126
--webserver_port=0
--builtin_ntp_servers=127.11.14.84:39431
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.14.126:33069
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 18:37:17.737737 31602 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:37:17.738022 31602 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:37:17.738057 31602 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:37:17.742004 31602 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:37:17.742059 31602 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:37:17.742075 31602 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:37:17.742091 31602 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:37:17.742106 31602 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:37:17.746686 31602 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.14.84:39431
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786127579900523-11321-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786127579900523-11321-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.14.126:33069
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.14.126:33069
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786127579900523-11321-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.14.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31602
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786127579900523-11321-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:37:17.747725 31602 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:37:17.749058 31602 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:37:17.757668 31607 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:37:17.757763 31608 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:37:17.757750 31602 server_base.cc:1034] running on GCE node
W20260807 18:37:17.757694 31610 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:37:17.758713 31602 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:37:17.759933 31602 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:37:17.761134 31602 hybrid_clock.cc:648] HybridClock initialized: now 1786127837761106 us; error 39 us; skew 500 ppm
I20260807 18:37:17.763085 31602 webserver.cc:460] Webserver started at http://127.11.14.126:39499/ using document root <none> and password file <none>
I20260807 18:37:17.763762 31602 fs_manager.cc:359] Metadata directory not provided
I20260807 18:37:17.763860 31602 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:37:17.764104 31602 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:37:17.766240 31602 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786127579900523-11321-0/minicluster-data/master-0/data/instance:
uuid: "cb005dd10bc14bcfabc73441ed20be9d"
format_stamp: "Formatted at 2026-08-07 18:37:17 on dist-test-slave-6k22"
server_key: "c7ddaa900c9c7d0db56417549e9e3f9a"
server_key_iv: "cea66ee70cbfa22c7a6a9b649af8681e"
server_key_version: "encryptionkey@0"
I20260807 18:37:17.766836 31602 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786127579900523-11321-0/minicluster-data/master-0/wal/instance:
uuid: "cb005dd10bc14bcfabc73441ed20be9d"
format_stamp: "Formatted at 2026-08-07 18:37:17 on dist-test-slave-6k22"
server_key: "c7ddaa900c9c7d0db56417549e9e3f9a"
server_key_iv: "cea66ee70cbfa22c7a6a9b649af8681e"
server_key_version: "encryptionkey@0"
I20260807 18:37:17.770617 31602 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.002s
I20260807 18:37:17.773186 31617 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:37:17.774410 31602 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 18:37:17.774561 31602 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786127579900523-11321-0/minicluster-data/master-0/data,/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786127579900523-11321-0/minicluster-data/master-0/wal
uuid: "cb005dd10bc14bcfabc73441ed20be9d"
format_stamp: "Formatted at 2026-08-07 18:37:17 on dist-test-slave-6k22"
server_key: "c7ddaa900c9c7d0db56417549e9e3f9a"
server_key_iv: "cea66ee70cbfa22c7a6a9b649af8681e"
server_key_version: "encryptionkey@0"
I20260807 18:37:17.774674 31602 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786127579900523-11321-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786127579900523-11321-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786127579900523-11321-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:37:17.788128 31602 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:37:17.788774 31602 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:37:17.788966 31602 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:37:17.796128 31602 rpc_server.cc:307] RPC server started. Bound to: 127.11.14.126:33069
I20260807 18:37:17.796159 31677 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.14.126:33069 every 8 connection(s)
I20260807 18:37:17.797078 31602 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786127579900523-11321-0/minicluster-data/master-0/data/info.pb
I20260807 18:37:17.798591 11321 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkz56XN/build/debug/bin/kudu as pid 31602
I20260807 18:37:17.798725 11321 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786127579900523-11321-0/minicluster-data/master-0/wal/instance
I20260807 18:37:17.799058 11321 external_mini_cluster.cc:1428] Setting key edf780ba26b657279f4e3d7eb4b415b0
I20260807 18:37:17.800743 31678 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:37:17.806197 31678 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cb005dd10bc14bcfabc73441ed20be9d: Bootstrap starting.
I20260807 18:37:17.808403 31681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:17.800638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59192 (local address 127.11.14.126:33069)
0807 18:37:17.801294 (+ 656us) server_negotiation.cc:206] Beginning negotiation
0807 18:37:17.801309 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 18:37:17.801344 (+ 35us) server_negotiation.cc:407] Connection header received
0807 18:37:17.801972 (+ 628us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:37:17.801988 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:37:17.802369 (+ 381us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:37:17.802704 (+ 335us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:37:17.803147 (+ 443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:17.804198 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:17.805111 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:17.805580 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:17.805765 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:17.806684 (+ 919us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:37:17.806689 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:37:17.806701 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:37:17.807071 (+ 370us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:37:17.807125 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:37:17.807144 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:37:17.807255 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0807 18:37:17.807390 (+ 135us) server_negotiation.cc:299] Negotiation successful
0807 18:37:17.807706 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":306,"thread_start_us":161,"threads_started":1}
I20260807 18:37:17.808487 31678 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cb005dd10bc14bcfabc73441ed20be9d: Neither blocks nor log segments found. Creating new log.
I20260807 18:37:17.809240 31678 log.cc:824] T 00000000000000000000000000000000 P cb005dd10bc14bcfabc73441ed20be9d: Log is configured to *not* fsync() on all Append() calls
I20260807 18:37:17.811306 31678 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cb005dd10bc14bcfabc73441ed20be9d: No bootstrap required, opened a new log
I20260807 18:37:17.814920 31678 raft_consensus.cc:348] T 00000000000000000000000000000000 P cb005dd10bc14bcfabc73441ed20be9d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb005dd10bc14bcfabc73441ed20be9d" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 33069 } }
I20260807 18:37:17.815150 31678 raft_consensus.cc:374] T 00000000000000000000000000000000 P cb005dd10bc14bcfabc73441ed20be9d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:37:17.815189 31678 raft_consensus.cc:729] T 00000000000000000000000000000000 P cb005dd10bc14bcfabc73441ed20be9d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb005dd10bc14bcfabc73441ed20be9d, State: Initialized, Role: FOLLOWER
I20260807 18:37:17.815622 31678 consensus_queue.cc:260] T 00000000000000000000000000000000 P cb005dd10bc14bcfabc73441ed20be9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb005dd10bc14bcfabc73441ed20be9d" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 33069 } }
I20260807 18:37:17.815745 31678 raft_consensus.cc:388] T 00000000000000000000000000000000 P cb005dd10bc14bcfabc73441ed20be9d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:37:17.815815 31678 raft_consensus.cc:482] T 00000000000000000000000000000000 P cb005dd10bc14bcfabc73441ed20be9d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:37:17.815886 31678 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cb005dd10bc14bcfabc73441ed20be9d [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:37:17.816705 31678 raft_consensus.cc:504] T 00000000000000000000000000000000 P cb005dd10bc14bcfabc73441ed20be9d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb005dd10bc14bcfabc73441ed20be9d" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 33069 } }
I20260807 18:37:17.817025 31678 leader_election.cc:302] T 00000000000000000000000000000000 P cb005dd10bc14bcfabc73441ed20be9d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cb005dd10bc14bcfabc73441ed20be9d; no voters:
I20260807 18:37:17.817276 31678 leader_election.cc:288] T 00000000000000000000000000000000 P cb005dd10bc14bcfabc73441ed20be9d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:37:17.817410 31683 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cb005dd10bc14bcfabc73441ed20be9d [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:37:17.817652 31683 raft_consensus.cc:686] T 00000000000000000000000000000000 P cb005dd10bc14bcfabc73441ed20be9d [term 1 LEADER]: Becoming Leader. State: Replica: cb005dd10bc14bcfabc73441ed20be9d, State: Running, Role: LEADER
I20260807 18:37:17.817940 31683 consensus_queue.cc:237] T 00000000000000000000000000000000 P cb005dd10bc14bcfabc73441ed20be9d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb005dd10bc14bcfabc73441ed20be9d" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 33069 } }
I20260807 18:37:17.818614 31678 sys_catalog.cc:564] T 00000000000000000000000000000000 P cb005dd10bc14bcfabc73441ed20be9d [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:37:17.819782 31684 sys_catalog.cc:455] T 00000000000000000000000000000000 P cb005dd10bc14bcfabc73441ed20be9d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cb005dd10bc14bcfabc73441ed20be9d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb005dd10bc14bcfabc73441ed20be9d" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 33069 } } }
I20260807 18:37:17.819834 31685 sys_catalog.cc:455] T 00000000000000000000000000000000 P cb005dd10bc14bcfabc73441ed20be9d [sys.catalog]: SysCatalogTable state changed. Reason: New leader cb005dd10bc14bcfabc73441ed20be9d. Latest consensus state: current_term: 1 leader_uuid: "cb005dd10bc14bcfabc73441ed20be9d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb005dd10bc14bcfabc73441ed20be9d" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 33069 } } }
I20260807 18:37:17.819922 31684 sys_catalog.cc:458] T 00000000000000000000000000000000 P cb005dd10bc14bcfabc73441ed20be9d [sys.catalog]: This master's current role is: LEADER
I20260807 18:37:17.819934 31685 sys_catalog.cc:458] T 00000000000000000000000000000000 P cb005dd10bc14bcfabc73441ed20be9d [sys.catalog]: This master's current role is: LEADER
I20260807 18:37:17.820312 31692 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:37:17.823352 31692 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:37:17.828679 31692 catalog_manager.cc:1347] Generated new cluster ID: 85e045706b054c5483aa96eb4507fa28
I20260807 18:37:17.828747 31692 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:37:17.847785 31692 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:37:17.848745 31692 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:37:17.865330 31692 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cb005dd10bc14bcfabc73441ed20be9d: Generated new TSK 0
I20260807 18:37:17.866109 31692 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:37:17.877202 11321 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkz56XN/build/debug/bin/kudu
/tmp/dist-test-taskkz56XN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786127579900523-11321-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786127579900523-11321-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786127579900523-11321-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786127579900523-11321-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.14.65:0
--local_ip_for_outbound_sockets=127.11.14.65
--webserver_interface=127.11.14.65
--webserver_port=0
--tserver_master_addrs=127.11.14.126:33069
--builtin_ntp_servers=127.11.14.84:39431
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 18:37:17.992226 31702 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:37:17.992524 31702 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:37:17.992596 31702 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:37:17.996538 31702 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:37:17.996619 31702 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:37:17.996711 31702 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.14.65
I20260807 18:37:18.001434 31702 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.14.84:39431
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786127579900523-11321-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786127579900523-11321-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.14.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786127579900523-11321-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.14.65
--webserver_port=0
--tserver_master_addrs=127.11.14.126:33069
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31702
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.14.65
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786127579900523-11321-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:37:18.002558 31702 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:37:18.004012 31702 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:37:18.013721 31708 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:37:18.013772 31711 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:37:18.013795 31709 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:37:18.013978 31702 server_base.cc:1034] running on GCE node
I20260807 18:37:18.014436 31702 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:37:18.015094 31702 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:37:18.016280 31702 hybrid_clock.cc:648] HybridClock initialized: now 1786127838016263 us; error 25 us; skew 500 ppm
I20260807 18:37:18.018081 31702 webserver.cc:460] Webserver started at http://127.11.14.65:34489/ using document root <none> and password file <none>
I20260807 18:37:18.018702 31702 fs_manager.cc:359] Metadata directory not provided
I20260807 18:37:18.018859 31702 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:37:18.019109 31702 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:37:18.021453 31702 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786127579900523-11321-0/minicluster-data/ts-0/data/instance:
uuid: "ddf7999748f64377a08d3d72110c2cec"
format_stamp: "Formatted at 2026-08-07 18:37:18 on dist-test-slave-6k22"
server_key: "19405a5107f9a8e417c85b51593cab6d"
server_key_iv: "59ffa3467b2a0671fa52a34271bfd114"
server_key_version: "encryptionkey@0"
I20260807 18:37:18.022035 31702 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786127579900523-11321-0/minicluster-data/ts-0/wal/instance:
uuid: "ddf7999748f64377a08d3d72110c2cec"
format_stamp: "Formatted at 2026-08-07 18:37:18 on dist-test-slave-6k22"
server_key: "19405a5107f9a8e417c85b51593cab6d"
server_key_iv: "59ffa3467b2a0671fa52a34271bfd114"
server_key_version: "encryptionkey@0"
I20260807 18:37:18.025801 31702 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260807 18:37:18.028281 31717 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:37:18.029567 31702 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 18:37:18.029706 31702 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786127579900523-11321-0/minicluster-data/ts-0/data,/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786127579900523-11321-0/minicluster-data/ts-0/wal
uuid: "ddf7999748f64377a08d3d72110c2cec"
format_stamp: "Formatted at 2026-08-07 18:37:18 on dist-test-slave-6k22"
server_key: "19405a5107f9a8e417c85b51593cab6d"
server_key_iv: "59ffa3467b2a0671fa52a34271bfd114"
server_key_version: "encryptionkey@0"
I20260807 18:37:18.029866 31702 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786127579900523-11321-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786127579900523-11321-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786127579900523-11321-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:37:18.043061 31702 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:37:18.043766 31702 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:37:18.043959 31702 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:37:18.044569 31702 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:37:18.045538 31702 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:37:18.045610 31702 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:37:18.045679 31702 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:37:18.045719 31702 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:37:18.054561 31702 rpc_server.cc:307] RPC server started. Bound to: 127.11.14.65:44677
I20260807 18:37:18.054585 31840 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.14.65:44677 every 8 connection(s)
I20260807 18:37:18.055524 31702 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786127579900523-11321-0/minicluster-data/ts-0/data/info.pb
I20260807 18:37:18.065055 11321 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkz56XN/build/debug/bin/kudu as pid 31702
I20260807 18:37:18.065214 11321 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786127579900523-11321-0/minicluster-data/ts-0/wal/instance
I20260807 18:37:18.065217 31681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:18.057368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.65:48939 (local address 127.11.14.126:33069)
0807 18:37:18.057507 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0807 18:37:18.057511 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:37:18.058579 (+ 1068us) server_negotiation.cc:407] Connection header received
0807 18:37:18.060204 (+ 1625us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:37:18.060207 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:37:18.060265 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:37:18.060386 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:37:18.061324 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:18.061890 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:18.063034 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:18.063269 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:18.063388 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:18.064569 (+ 1181us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:37:18.064577 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:37:18.064579 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:37:18.064590 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:37:18.064609 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:37:18.064617 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:37:18.064694 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0807 18:37:18.064991 (+ 297us) server_negotiation.cc:299] Negotiation successful
0807 18:37:18.065056 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260807 18:37:18.065529 11321 external_mini_cluster.cc:1428] Setting key 336a707b2dd382ce3de2717b73168147
I20260807 18:37:18.066180 31843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:18.057775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:33069 (local address 127.11.14.65:48939)
0807 18:37:18.058424 (+ 649us) negotiation.cc:107] Waiting for socket to connect
0807 18:37:18.058465 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 18:37:18.059990 (+ 1525us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:37:18.060567 (+ 577us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:37:18.060574 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:37:18.060657 (+ 83us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:37:18.061175 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:37:18.061188 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:37:18.062028 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:37:18.062031 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:37:18.062901 (+ 870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:37:18.062911 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:37:18.063433 (+ 522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:37:18.063437 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:37:18.063661 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:18.064298 (+ 637us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 18:37:18.064300 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:37:18.064341 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 18:37:18.064403 (+ 62us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:37:18.064696 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:37:18.064709 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:37:18.064715 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 18:37:18.064889 (+ 174us) client_negotiation.cc:240] Negotiation successful
0807 18:37:18.065198 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":107,"threads_started":1}
I20260807 18:37:18.067431 31841 heartbeater.cc:347] Connected to a master server at 127.11.14.126:33069
I20260807 18:37:18.067703 31841 heartbeater.cc:464] Registering TS with master...
I20260807 18:37:18.067679 11321 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkz56XN/build/debug/bin/kudu
/tmp/dist-test-taskkz56XN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786127579900523-11321-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786127579900523-11321-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786127579900523-11321-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786127579900523-11321-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.14.66:0
--local_ip_for_outbound_sockets=127.11.14.66
--webserver_interface=127.11.14.66
--webserver_port=0
--tserver_master_addrs=127.11.14.126:33069
--builtin_ntp_servers=127.11.14.84:39431
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 18:37:18.068315 31841 heartbeater.cc:511] Master 127.11.14.126:33069 requested a full tablet report, sending...
I20260807 18:37:18.069947 31634 ts_manager.cc:194] Registered new tserver with Master: ddf7999748f64377a08d3d72110c2cec (127.11.14.65:44677)
I20260807 18:37:18.071056 31634 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.14.65:48939
W20260807 18:37:18.187227 31844 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:37:18.187485 31844 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:37:18.187523 31844 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:37:18.191493 31844 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:37:18.191545 31844 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:37:18.191686 31844 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.14.66
I20260807 18:37:18.196483 31844 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.14.84:39431
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786127579900523-11321-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786127579900523-11321-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.14.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786127579900523-11321-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.14.66
--webserver_port=0
--tserver_master_addrs=127.11.14.126:33069
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31844
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.14.66
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786127579900523-11321-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:37:18.197535 31844 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:37:18.198943 31844 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:37:18.208782 31853 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:37:18.208789 31850 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:37:18.208824 31851 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:37:18.209080 31844 server_base.cc:1034] running on GCE node
I20260807 18:37:18.209568 31844 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:37:18.210250 31844 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:37:18.211428 31844 hybrid_clock.cc:648] HybridClock initialized: now 1786127838211394 us; error 45 us; skew 500 ppm
I20260807 18:37:18.213538 31844 webserver.cc:460] Webserver started at http://127.11.14.66:40341/ using document root <none> and password file <none>
I20260807 18:37:18.214195 31844 fs_manager.cc:359] Metadata directory not provided
I20260807 18:37:18.214267 31844 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:37:18.214541 31844 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:37:18.216867 31844 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786127579900523-11321-0/minicluster-data/ts-1/data/instance:
uuid: "386df41d14874f7dbcf5d9e2044b7a04"
format_stamp: "Formatted at 2026-08-07 18:37:18 on dist-test-slave-6k22"
server_key: "54f17da89a2492cb7d3e6620dae11297"
server_key_iv: "86bac0269a7d441e0e167eb98e05edb2"
server_key_version: "encryptionkey@0"
I20260807 18:37:18.217499 31844 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786127579900523-11321-0/minicluster-data/ts-1/wal/instance:
uuid: "386df41d14874f7dbcf5d9e2044b7a04"
format_stamp: "Formatted at 2026-08-07 18:37:18 on dist-test-slave-6k22"
server_key: "54f17da89a2492cb7d3e6620dae11297"
server_key_iv: "86bac0269a7d441e0e167eb98e05edb2"
server_key_version: "encryptionkey@0"
I20260807 18:37:18.222048 31844 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 18:37:18.224973 31860 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:37:18.226514 31844 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:37:18.226648 31844 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786127579900523-11321-0/minicluster-data/ts-1/data,/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786127579900523-11321-0/minicluster-data/ts-1/wal
uuid: "386df41d14874f7dbcf5d9e2044b7a04"
format_stamp: "Formatted at 2026-08-07 18:37:18 on dist-test-slave-6k22"
server_key: "54f17da89a2492cb7d3e6620dae11297"
server_key_iv: "86bac0269a7d441e0e167eb98e05edb2"
server_key_version: "encryptionkey@0"
I20260807 18:37:18.226776 31844 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786127579900523-11321-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786127579900523-11321-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786127579900523-11321-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:37:18.241035 31844 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:37:18.241792 31844 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:37:18.241993 31844 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:37:18.242583 31844 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:37:18.243520 31844 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:37:18.243613 31844 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:37:18.243700 31844 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:37:18.243753 31844 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:37:18.252867 31844 rpc_server.cc:307] RPC server started. Bound to: 127.11.14.66:38559
I20260807 18:37:18.252977 31984 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.14.66:38559 every 8 connection(s)
I20260807 18:37:18.254047 31844 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786127579900523-11321-0/minicluster-data/ts-1/data/info.pb
I20260807 18:37:18.255591 11321 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkz56XN/build/debug/bin/kudu as pid 31844
I20260807 18:37:18.255755 11321 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786127579900523-11321-0/minicluster-data/ts-1/wal/instance
I20260807 18:37:18.256129 11321 external_mini_cluster.cc:1428] Setting key 7edb5782b00eb8e157144c0af0cb38bd
I20260807 18:37:18.258498 11321 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkz56XN/build/debug/bin/kudu
/tmp/dist-test-taskkz56XN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786127579900523-11321-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786127579900523-11321-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786127579900523-11321-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786127579900523-11321-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.14.67:0
--local_ip_for_outbound_sockets=127.11.14.67
--webserver_interface=127.11.14.67
--webserver_port=0
--tserver_master_addrs=127.11.14.126:33069
--builtin_ntp_servers=127.11.14.84:39431
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 18:37:18.264930 31681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:18.256339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.66:59979 (local address 127.11.14.126:33069)
0807 18:37:18.256504 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0807 18:37:18.256508 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:37:18.257765 (+ 1257us) server_negotiation.cc:407] Connection header received
0807 18:37:18.259299 (+ 1534us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:37:18.259302 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:37:18.259359 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:37:18.259438 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:37:18.260597 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:18.261122 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:18.262772 (+ 1650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:18.262992 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:18.263073 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:18.264435 (+ 1362us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:37:18.264438 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:37:18.264440 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:37:18.264452 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:37:18.264473 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:37:18.264480 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:37:18.264541 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0807 18:37:18.264748 (+ 207us) server_negotiation.cc:299] Negotiation successful
0807 18:37:18.264805 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260807 18:37:18.266198 31987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:18.256814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:33069 (local address 127.11.14.66:59979)
0807 18:37:18.257573 (+ 759us) negotiation.cc:107] Waiting for socket to connect
0807 18:37:18.257638 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0807 18:37:18.259076 (+ 1438us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:37:18.259673 (+ 597us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:37:18.259681 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:37:18.259769 (+ 88us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:37:18.260421 (+ 652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:37:18.260443 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:37:18.261257 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:37:18.261259 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:37:18.262611 (+ 1352us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:37:18.262627 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:37:18.263160 (+ 533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:37:18.263168 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:37:18.263422 (+ 254us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:18.264116 (+ 694us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 18:37:18.264120 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:37:18.264180 (+ 60us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 18:37:18.264271 (+ 91us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:37:18.264571 (+ 300us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:37:18.264584 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:37:18.264593 (+ 9us) client_negotiation.cc:769] Sending connection context
0807 18:37:18.264747 (+ 154us) client_negotiation.cc:240] Negotiation successful
0807 18:37:18.265071 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":288,"thread_start_us":124,"threads_started":1}
I20260807 18:37:18.267416 31985 heartbeater.cc:347] Connected to a master server at 127.11.14.126:33069
I20260807 18:37:18.267702 31985 heartbeater.cc:464] Registering TS with master...
I20260807 18:37:18.268217 31985 heartbeater.cc:511] Master 127.11.14.126:33069 requested a full tablet report, sending...
I20260807 18:37:18.269304 31634 ts_manager.cc:194] Registered new tserver with Master: 386df41d14874f7dbcf5d9e2044b7a04 (127.11.14.66:38559)
I20260807 18:37:18.269878 31634 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.14.66:59979
W20260807 18:37:18.376505 31988 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:37:18.376794 31988 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:37:18.376874 31988 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:37:18.380966 31988 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:37:18.381054 31988 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:37:18.381146 31988 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.14.67
I20260807 18:37:18.385950 31988 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.14.84:39431
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786127579900523-11321-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786127579900523-11321-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.14.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786127579900523-11321-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.14.67
--webserver_port=0
--tserver_master_addrs=127.11.14.126:33069
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31988
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.14.67
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786127579900523-11321-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:37:18.387087 31988 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:37:18.388557 31988 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:37:18.398537 31994 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:37:18.398532 31996 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:37:18.398535 31993 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:37:18.398741 31988 server_base.cc:1034] running on GCE node
I20260807 18:37:18.399354 31988 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:37:18.400099 31988 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:37:18.401300 31988 hybrid_clock.cc:648] HybridClock initialized: now 1786127838401277 us; error 35 us; skew 500 ppm
I20260807 18:37:18.403141 31988 webserver.cc:460] Webserver started at http://127.11.14.67:45245/ using document root <none> and password file <none>
I20260807 18:37:18.403867 31988 fs_manager.cc:359] Metadata directory not provided
I20260807 18:37:18.403973 31988 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:37:18.404218 31988 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:37:18.406329 31988 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786127579900523-11321-0/minicluster-data/ts-2/data/instance:
uuid: "4d5ced83b17f41d6958772a8c45eed1b"
format_stamp: "Formatted at 2026-08-07 18:37:18 on dist-test-slave-6k22"
server_key: "e35c24368e4af05ecf07f9a246a5c5db"
server_key_iv: "c345047c808d203526a3e380ef8de41b"
server_key_version: "encryptionkey@0"
I20260807 18:37:18.406921 31988 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786127579900523-11321-0/minicluster-data/ts-2/wal/instance:
uuid: "4d5ced83b17f41d6958772a8c45eed1b"
format_stamp: "Formatted at 2026-08-07 18:37:18 on dist-test-slave-6k22"
server_key: "e35c24368e4af05ecf07f9a246a5c5db"
server_key_iv: "c345047c808d203526a3e380ef8de41b"
server_key_version: "encryptionkey@0"
I20260807 18:37:18.410694 31988 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260807 18:37:18.413362 32002 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:37:18.414853 31988 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 18:37:18.414968 31988 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786127579900523-11321-0/minicluster-data/ts-2/data,/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786127579900523-11321-0/minicluster-data/ts-2/wal
uuid: "4d5ced83b17f41d6958772a8c45eed1b"
format_stamp: "Formatted at 2026-08-07 18:37:18 on dist-test-slave-6k22"
server_key: "e35c24368e4af05ecf07f9a246a5c5db"
server_key_iv: "c345047c808d203526a3e380ef8de41b"
server_key_version: "encryptionkey@0"
I20260807 18:37:18.415068 31988 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786127579900523-11321-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786127579900523-11321-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786127579900523-11321-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:37:18.440140 31988 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:37:18.440883 31988 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:37:18.441048 31988 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:37:18.441638 31988 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:37:18.442647 31988 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:37:18.442696 31988 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:37:18.442737 31988 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:37:18.442756 31988 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:37:18.452145 31988 rpc_server.cc:307] RPC server started. Bound to: 127.11.14.67:37067
I20260807 18:37:18.452222 32129 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.14.67:37067 every 8 connection(s)
I20260807 18:37:18.453140 31988 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786127579900523-11321-0/minicluster-data/ts-2/data/info.pb
I20260807 18:37:18.456790 11321 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkz56XN/build/debug/bin/kudu as pid 31988
I20260807 18:37:18.456945 11321 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786127579900523-11321-0/minicluster-data/ts-2/wal/instance
I20260807 18:37:18.457224 11321 external_mini_cluster.cc:1428] Setting key c9760e1ca460da74e52dd3886c8feff1
I20260807 18:37:18.463178 31681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:18.455090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.67:36141 (local address 127.11.14.126:33069)
0807 18:37:18.455243 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0807 18:37:18.455247 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:37:18.456385 (+ 1138us) server_negotiation.cc:407] Connection header received
0807 18:37:18.458242 (+ 1857us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:37:18.458246 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:37:18.458305 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:37:18.458417 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:37:18.459545 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:18.460107 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:18.461268 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:18.461496 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:18.461611 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:18.462662 (+ 1051us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:37:18.462666 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:37:18.462668 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:37:18.462688 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:37:18.462707 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:37:18.462714 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:37:18.462820 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0807 18:37:18.462978 (+ 158us) server_negotiation.cc:299] Negotiation successful
0807 18:37:18.463040 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260807 18:37:18.464099 32132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:18.455490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:33069 (local address 127.11.14.67:36141)
0807 18:37:18.456223 (+ 733us) negotiation.cc:107] Waiting for socket to connect
0807 18:37:18.456265 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 18:37:18.458003 (+ 1738us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:37:18.458599 (+ 596us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:37:18.458605 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:37:18.458710 (+ 105us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:37:18.459407 (+ 697us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:37:18.459424 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:37:18.460239 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:37:18.460243 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:37:18.461141 (+ 898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:37:18.461151 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:37:18.461653 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:37:18.461657 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:37:18.461838 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:18.462408 (+ 570us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 18:37:18.462409 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:37:18.462450 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 18:37:18.462512 (+ 62us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:37:18.462814 (+ 302us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:37:18.462825 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:37:18.462833 (+ 8us) client_negotiation.cc:769] Sending connection context
0807 18:37:18.462977 (+ 144us) client_negotiation.cc:240] Negotiation successful
0807 18:37:18.463262 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":331,"thread_start_us":116,"threads_started":1}
I20260807 18:37:18.465179 32130 heartbeater.cc:347] Connected to a master server at 127.11.14.126:33069
I20260807 18:37:18.465407 32130 heartbeater.cc:464] Registering TS with master...
I20260807 18:37:18.465880 32130 heartbeater.cc:511] Master 127.11.14.126:33069 requested a full tablet report, sending...
I20260807 18:37:18.466868 31634 ts_manager.cc:194] Registered new tserver with Master: 4d5ced83b17f41d6958772a8c45eed1b (127.11.14.67:37067)
I20260807 18:37:18.467383 31634 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.14.67:36141
I20260807 18:37:18.471181 11321 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 18:37:18.476725 31681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:18.473211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38392 (local address 127.11.14.126:33069)
0807 18:37:18.473342 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0807 18:37:18.473347 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:37:18.473493 (+ 146us) server_negotiation.cc:407] Connection header received
0807 18:37:18.473589 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:37:18.473591 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:37:18.473632 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:37:18.473699 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:37:18.473947 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:18.474441 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:18.475212 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:18.475421 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:18.475494 (+ 73us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:18.476315 (+ 821us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:37:18.476318 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:37:18.476320 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:37:18.476332 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:37:18.476350 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:37:18.476357 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:37:18.476458 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0807 18:37:18.476590 (+ 132us) server_negotiation.cc:299] Negotiation successful
0807 18:37:18.476621 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260807 18:37:18.479823 31634 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38392:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 18:37:18.482780 31634 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 18:37:18.497938 32143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:18.491786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60904 (local address 127.11.14.65:44677)
0807 18:37:18.492450 (+ 664us) server_negotiation.cc:206] Beginning negotiation
0807 18:37:18.492463 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 18:37:18.492491 (+ 28us) server_negotiation.cc:407] Connection header received
0807 18:37:18.492679 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:37:18.492685 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:37:18.492927 (+ 242us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:37:18.493093 (+ 166us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:37:18.493602 (+ 509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:18.494835 (+ 1233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:18.496734 (+ 1899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:18.497268 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:18.497402 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:18.497611 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0807 18:37:18.497671 (+ 60us) server_negotiation.cc:299] Negotiation successful
0807 18:37:18.497762 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":554,"thread_start_us":108,"threads_started":1}
I20260807 18:37:18.497926 32142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:18.491679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.65:44677 (local address 127.0.0.1:60904)
0807 18:37:18.492288 (+ 609us) negotiation.cc:107] Waiting for socket to connect
0807 18:37:18.492339 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0807 18:37:18.492461 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:37:18.493283 (+ 822us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:37:18.493289 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:37:18.493314 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:37:18.493482 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:37:18.493490 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:37:18.494992 (+ 1502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:37:18.494997 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:37:18.496610 (+ 1613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:37:18.496616 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:37:18.497402 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:37:18.497406 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:37:18.497509 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:18.497545 (+ 36us) client_negotiation.cc:769] Sending connection context
0807 18:37:18.497647 (+ 102us) client_negotiation.cc:240] Negotiation successful
0807 18:37:18.497762 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":435,"mutex_wait_us":60,"thread_start_us":72,"threads_started":1}
I20260807 18:37:18.498390 32144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:18.492124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48262 (local address 127.11.14.67:37067)
0807 18:37:18.492548 (+ 424us) server_negotiation.cc:206] Beginning negotiation
0807 18:37:18.492559 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 18:37:18.492855 (+ 296us) server_negotiation.cc:407] Connection header received
0807 18:37:18.492943 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:37:18.492949 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:37:18.493154 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:37:18.493296 (+ 142us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:37:18.493693 (+ 397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:18.494835 (+ 1142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:18.496549 (+ 1714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:18.497360 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:18.497471 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:18.497672 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0807 18:37:18.498098 (+ 426us) server_negotiation.cc:299] Negotiation successful
0807 18:37:18.498194 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":328,"thread_start_us":98,"threads_started":1}
I20260807 18:37:18.498587 32146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:18.492830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55596 (local address 127.11.14.66:38559)
0807 18:37:18.493286 (+ 456us) server_negotiation.cc:206] Beginning negotiation
0807 18:37:18.493298 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 18:37:18.493703 (+ 405us) server_negotiation.cc:407] Connection header received
0807 18:37:18.493786 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:37:18.493789 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:37:18.493918 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:37:18.494067 (+ 149us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:37:18.494267 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:18.495075 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:18.496715 (+ 1640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:18.497454 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:18.497542 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:18.497682 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0807 18:37:18.498304 (+ 622us) server_negotiation.cc:299] Negotiation successful
0807 18:37:18.498419 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":347,"thread_start_us":144,"threads_started":1}
I20260807 18:37:18.498986 32141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:18.491734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.67:37067 (local address 127.0.0.1:48262)
0807 18:37:18.492628 (+ 894us) negotiation.cc:107] Waiting for socket to connect
0807 18:37:18.492642 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 18:37:18.492740 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:37:18.493375 (+ 635us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:37:18.493378 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:37:18.493396 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:37:18.493511 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:37:18.493517 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:37:18.494961 (+ 1444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:37:18.494965 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:37:18.496427 (+ 1462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:37:18.496437 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:37:18.497740 (+ 1303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:37:18.497744 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:37:18.497873 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:18.497924 (+ 51us) client_negotiation.cc:769] Sending connection context
0807 18:37:18.498030 (+ 106us) client_negotiation.cc:240] Negotiation successful
0807 18:37:18.498824 (+ 794us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":831,"mutex_wait_us":838,"thread_start_us":110,"threads_started":1}
I20260807 18:37:18.498991 32145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:18.492434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.66:38559 (local address 127.0.0.1:55596)
0807 18:37:18.493537 (+ 1103us) negotiation.cc:107] Waiting for socket to connect
0807 18:37:18.493552 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 18:37:18.493639 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:37:18.494048 (+ 409us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:37:18.494051 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:37:18.494068 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:37:18.494143 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:37:18.494148 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:37:18.495190 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:37:18.495193 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:37:18.496587 (+ 1394us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:37:18.496597 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:37:18.497740 (+ 1143us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:37:18.497744 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:37:18.497888 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:18.497942 (+ 54us) client_negotiation.cc:769] Sending connection context
0807 18:37:18.498005 (+ 63us) client_negotiation.cc:240] Negotiation successful
0807 18:37:18.498879 (+ 874us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1041,"mutex_wait_us":392,"thread_start_us":90,"threads_started":1}
I20260807 18:37:18.501392 31770 tablet_service.cc:1467] Processing CreateTablet for tablet add1286a898f4f75b7d534e37181cb86 (DEFAULT_TABLE table=test-table [id=0bb083847c8649d49a7addac1d37eeca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:37:18.501586 31911 tablet_service.cc:1467] Processing CreateTablet for tablet add1286a898f4f75b7d534e37181cb86 (DEFAULT_TABLE table=test-table [id=0bb083847c8649d49a7addac1d37eeca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:37:18.501878 32054 tablet_service.cc:1467] Processing CreateTablet for tablet add1286a898f4f75b7d534e37181cb86 (DEFAULT_TABLE table=test-table [id=0bb083847c8649d49a7addac1d37eeca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:37:18.502526 31911 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet add1286a898f4f75b7d534e37181cb86. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:37:18.502538 31770 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet add1286a898f4f75b7d534e37181cb86. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:37:18.502736 32054 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet add1286a898f4f75b7d534e37181cb86. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:37:18.508589 32147 tablet_bootstrap.cc:492] T add1286a898f4f75b7d534e37181cb86 P 386df41d14874f7dbcf5d9e2044b7a04: Bootstrap starting.
I20260807 18:37:18.510376 32149 tablet_bootstrap.cc:492] T add1286a898f4f75b7d534e37181cb86 P ddf7999748f64377a08d3d72110c2cec: Bootstrap starting.
I20260807 18:37:18.511015 32147 tablet_bootstrap.cc:654] T add1286a898f4f75b7d534e37181cb86 P 386df41d14874f7dbcf5d9e2044b7a04: Neither blocks nor log segments found. Creating new log.
I20260807 18:37:18.511893 32147 log.cc:824] T add1286a898f4f75b7d534e37181cb86 P 386df41d14874f7dbcf5d9e2044b7a04: Log is configured to *not* fsync() on all Append() calls
I20260807 18:37:18.511922 32148 tablet_bootstrap.cc:492] T add1286a898f4f75b7d534e37181cb86 P 4d5ced83b17f41d6958772a8c45eed1b: Bootstrap starting.
I20260807 18:37:18.512478 32149 tablet_bootstrap.cc:654] T add1286a898f4f75b7d534e37181cb86 P ddf7999748f64377a08d3d72110c2cec: Neither blocks nor log segments found. Creating new log.
I20260807 18:37:18.513236 32149 log.cc:824] T add1286a898f4f75b7d534e37181cb86 P ddf7999748f64377a08d3d72110c2cec: Log is configured to *not* fsync() on all Append() calls
I20260807 18:37:18.514019 32148 tablet_bootstrap.cc:654] T add1286a898f4f75b7d534e37181cb86 P 4d5ced83b17f41d6958772a8c45eed1b: Neither blocks nor log segments found. Creating new log.
I20260807 18:37:18.514240 32147 tablet_bootstrap.cc:492] T add1286a898f4f75b7d534e37181cb86 P 386df41d14874f7dbcf5d9e2044b7a04: No bootstrap required, opened a new log
I20260807 18:37:18.514438 32147 ts_tablet_manager.cc:1397] T add1286a898f4f75b7d534e37181cb86 P 386df41d14874f7dbcf5d9e2044b7a04: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260807 18:37:18.514845 32148 log.cc:824] T add1286a898f4f75b7d534e37181cb86 P 4d5ced83b17f41d6958772a8c45eed1b: Log is configured to *not* fsync() on all Append() calls
I20260807 18:37:18.515671 32149 tablet_bootstrap.cc:492] T add1286a898f4f75b7d534e37181cb86 P ddf7999748f64377a08d3d72110c2cec: No bootstrap required, opened a new log
I20260807 18:37:18.515862 32149 ts_tablet_manager.cc:1397] T add1286a898f4f75b7d534e37181cb86 P ddf7999748f64377a08d3d72110c2cec: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.002s
I20260807 18:37:18.516868 32148 tablet_bootstrap.cc:492] T add1286a898f4f75b7d534e37181cb86 P 4d5ced83b17f41d6958772a8c45eed1b: No bootstrap required, opened a new log
I20260807 18:37:18.517102 32148 ts_tablet_manager.cc:1397] T add1286a898f4f75b7d534e37181cb86 P 4d5ced83b17f41d6958772a8c45eed1b: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260807 18:37:18.518975 32147 raft_consensus.cc:348] T add1286a898f4f75b7d534e37181cb86 P 386df41d14874f7dbcf5d9e2044b7a04 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ddf7999748f64377a08d3d72110c2cec" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 44677 } } peers { permanent_uuid: "4d5ced83b17f41d6958772a8c45eed1b" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 37067 } } peers { permanent_uuid: "386df41d14874f7dbcf5d9e2044b7a04" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 38559 } }
I20260807 18:37:18.519316 32147 raft_consensus.cc:374] T add1286a898f4f75b7d534e37181cb86 P 386df41d14874f7dbcf5d9e2044b7a04 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:37:18.519387 32147 raft_consensus.cc:729] T add1286a898f4f75b7d534e37181cb86 P 386df41d14874f7dbcf5d9e2044b7a04 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 386df41d14874f7dbcf5d9e2044b7a04, State: Initialized, Role: FOLLOWER
I20260807 18:37:18.519291 32149 raft_consensus.cc:348] T add1286a898f4f75b7d534e37181cb86 P ddf7999748f64377a08d3d72110c2cec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ddf7999748f64377a08d3d72110c2cec" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 44677 } } peers { permanent_uuid: "4d5ced83b17f41d6958772a8c45eed1b" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 37067 } } peers { permanent_uuid: "386df41d14874f7dbcf5d9e2044b7a04" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 38559 } }
I20260807 18:37:18.519582 32149 raft_consensus.cc:374] T add1286a898f4f75b7d534e37181cb86 P ddf7999748f64377a08d3d72110c2cec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:37:18.519665 32149 raft_consensus.cc:729] T add1286a898f4f75b7d534e37181cb86 P ddf7999748f64377a08d3d72110c2cec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ddf7999748f64377a08d3d72110c2cec, State: Initialized, Role: FOLLOWER
I20260807 18:37:18.519948 32147 consensus_queue.cc:260] T add1286a898f4f75b7d534e37181cb86 P 386df41d14874f7dbcf5d9e2044b7a04 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ddf7999748f64377a08d3d72110c2cec" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 44677 } } peers { permanent_uuid: "4d5ced83b17f41d6958772a8c45eed1b" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 37067 } } peers { permanent_uuid: "386df41d14874f7dbcf5d9e2044b7a04" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 38559 } }
I20260807 18:37:18.520155 32149 consensus_queue.cc:260] T add1286a898f4f75b7d534e37181cb86 P ddf7999748f64377a08d3d72110c2cec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ddf7999748f64377a08d3d72110c2cec" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 44677 } } peers { permanent_uuid: "4d5ced83b17f41d6958772a8c45eed1b" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 37067 } } peers { permanent_uuid: "386df41d14874f7dbcf5d9e2044b7a04" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 38559 } }
I20260807 18:37:18.520778 31985 heartbeater.cc:503] Master 127.11.14.126:33069 was elected leader, sending a full tablet report...
I20260807 18:37:18.521127 32149 ts_tablet_manager.cc:1428] T add1286a898f4f75b7d534e37181cb86 P ddf7999748f64377a08d3d72110c2cec: Time spent starting tablet: real 0.005s user 0.003s sys 0.002s
I20260807 18:37:18.521479 31841 heartbeater.cc:503] Master 127.11.14.126:33069 was elected leader, sending a full tablet report...
I20260807 18:37:18.521481 32148 raft_consensus.cc:348] T add1286a898f4f75b7d534e37181cb86 P 4d5ced83b17f41d6958772a8c45eed1b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ddf7999748f64377a08d3d72110c2cec" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 44677 } } peers { permanent_uuid: "4d5ced83b17f41d6958772a8c45eed1b" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 37067 } } peers { permanent_uuid: "386df41d14874f7dbcf5d9e2044b7a04" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 38559 } }
I20260807 18:37:18.521710 32148 raft_consensus.cc:374] T add1286a898f4f75b7d534e37181cb86 P 4d5ced83b17f41d6958772a8c45eed1b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:37:18.521761 32148 raft_consensus.cc:729] T add1286a898f4f75b7d534e37181cb86 P 4d5ced83b17f41d6958772a8c45eed1b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d5ced83b17f41d6958772a8c45eed1b, State: Initialized, Role: FOLLOWER
I20260807 18:37:18.522130 32148 consensus_queue.cc:260] T add1286a898f4f75b7d534e37181cb86 P 4d5ced83b17f41d6958772a8c45eed1b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ddf7999748f64377a08d3d72110c2cec" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 44677 } } peers { permanent_uuid: "4d5ced83b17f41d6958772a8c45eed1b" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 37067 } } peers { permanent_uuid: "386df41d14874f7dbcf5d9e2044b7a04" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 38559 } }
I20260807 18:37:18.522305 32147 ts_tablet_manager.cc:1428] T add1286a898f4f75b7d534e37181cb86 P 386df41d14874f7dbcf5d9e2044b7a04: Time spent starting tablet: real 0.008s user 0.008s sys 0.000s
I20260807 18:37:18.523002 32148 ts_tablet_manager.cc:1428] T add1286a898f4f75b7d534e37181cb86 P 4d5ced83b17f41d6958772a8c45eed1b: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260807 18:37:18.523077 32130 heartbeater.cc:503] Master 127.11.14.126:33069 was elected leader, sending a full tablet report...
W20260807 18:37:18.556811 31842 tablet.cc:2369] T add1286a898f4f75b7d534e37181cb86 P ddf7999748f64377a08d3d72110c2cec: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:37:18.625195 32156 raft_consensus.cc:482] T add1286a898f4f75b7d534e37181cb86 P 4d5ced83b17f41d6958772a8c45eed1b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:37:18.625475 32156 raft_consensus.cc:504] T add1286a898f4f75b7d534e37181cb86 P 4d5ced83b17f41d6958772a8c45eed1b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ddf7999748f64377a08d3d72110c2cec" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 44677 } } peers { permanent_uuid: "4d5ced83b17f41d6958772a8c45eed1b" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 37067 } } peers { permanent_uuid: "386df41d14874f7dbcf5d9e2044b7a04" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 38559 } }
I20260807 18:37:18.626868 32156 leader_election.cc:288] T add1286a898f4f75b7d534e37181cb86 P 4d5ced83b17f41d6958772a8c45eed1b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ddf7999748f64377a08d3d72110c2cec (127.11.14.65:44677), 386df41d14874f7dbcf5d9e2044b7a04 (127.11.14.66:38559)
I20260807 18:37:18.630569 32143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:18.627199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.67:53555 (local address 127.11.14.65:44677)
0807 18:37:18.627338 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0807 18:37:18.627342 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:37:18.627356 (+ 14us) server_negotiation.cc:407] Connection header received
0807 18:37:18.627513 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:37:18.627517 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:37:18.627608 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:37:18.627716 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:37:18.628016 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:18.628517 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:18.629545 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:18.629978 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:18.630117 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:18.630172 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0807 18:37:18.630367 (+ 195us) server_negotiation.cc:299] Negotiation successful
0807 18:37:18.630436 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260807 18:37:18.630569 32132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:18.627094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.65:44677 (local address 127.11.14.67:53555)
0807 18:37:18.627248 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0807 18:37:18.627267 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 18:37:18.627381 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:37:18.627733 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:37:18.627736 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:37:18.627756 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:37:18.627889 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:37:18.627896 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:37:18.628626 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:37:18.628632 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:37:18.629413 (+ 781us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:37:18.629422 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:37:18.630110 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:37:18.630114 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:37:18.630225 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:18.630253 (+ 28us) client_negotiation.cc:769] Sending connection context
0807 18:37:18.630351 (+ 98us) client_negotiation.cc:240] Negotiation successful
0807 18:37:18.630436 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":59}
I20260807 18:37:18.631222 31792 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "add1286a898f4f75b7d534e37181cb86" candidate_uuid: "4d5ced83b17f41d6958772a8c45eed1b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ddf7999748f64377a08d3d72110c2cec" is_pre_election: true
I20260807 18:37:18.631525 31792 raft_consensus.cc:2454] T add1286a898f4f75b7d534e37181cb86 P ddf7999748f64377a08d3d72110c2cec [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4d5ced83b17f41d6958772a8c45eed1b in term 0.
I20260807 18:37:18.632154 32157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:18.627450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.66:38559 (local address 127.11.14.67:38405)
0807 18:37:18.627980 (+ 530us) negotiation.cc:107] Waiting for socket to connect
0807 18:37:18.627992 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 18:37:18.628125 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:37:18.628411 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:37:18.628414 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:37:18.628434 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:37:18.628584 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:37:18.628590 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:37:18.629380 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:37:18.629384 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:37:18.630577 (+ 1193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:37:18.630594 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:37:18.631719 (+ 1125us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:37:18.631723 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:37:18.631810 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:18.631850 (+ 40us) client_negotiation.cc:769] Sending connection context
0807 18:37:18.631945 (+ 95us) client_negotiation.cc:240] Negotiation successful
0807 18:37:18.632023 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":471,"thread_start_us":218,"threads_started":1}
I20260807 18:37:18.632185 32146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:18.627682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.67:38405 (local address 127.11.14.66:38559)
0807 18:37:18.627841 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0807 18:37:18.627847 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:37:18.628077 (+ 230us) server_negotiation.cc:407] Connection header received
0807 18:37:18.628247 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:37:18.628250 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:37:18.628304 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:37:18.628397 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:37:18.628708 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:18.629254 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:18.630778 (+ 1524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:18.631400 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:18.631498 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:18.631544 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0807 18:37:18.631982 (+ 438us) server_negotiation.cc:299] Negotiation successful
0807 18:37:18.632059 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260807 18:37:18.632110 32005 leader_election.cc:302] T add1286a898f4f75b7d534e37181cb86 P 4d5ced83b17f41d6958772a8c45eed1b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4d5ced83b17f41d6958772a8c45eed1b, ddf7999748f64377a08d3d72110c2cec; no voters:
I20260807 18:37:18.632447 32156 raft_consensus.cc:2781] T add1286a898f4f75b7d534e37181cb86 P 4d5ced83b17f41d6958772a8c45eed1b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:37:18.632577 32156 raft_consensus.cc:482] T add1286a898f4f75b7d534e37181cb86 P 4d5ced83b17f41d6958772a8c45eed1b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:37:18.632681 32156 raft_consensus.cc:3037] T add1286a898f4f75b7d534e37181cb86 P 4d5ced83b17f41d6958772a8c45eed1b [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:37:18.632920 31933 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "add1286a898f4f75b7d534e37181cb86" candidate_uuid: "4d5ced83b17f41d6958772a8c45eed1b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "386df41d14874f7dbcf5d9e2044b7a04" is_pre_election: true
I20260807 18:37:18.633203 31933 raft_consensus.cc:2454] T add1286a898f4f75b7d534e37181cb86 P 386df41d14874f7dbcf5d9e2044b7a04 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4d5ced83b17f41d6958772a8c45eed1b in term 0.
I20260807 18:37:18.634119 32156 raft_consensus.cc:504] T add1286a898f4f75b7d534e37181cb86 P 4d5ced83b17f41d6958772a8c45eed1b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ddf7999748f64377a08d3d72110c2cec" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 44677 } } peers { permanent_uuid: "4d5ced83b17f41d6958772a8c45eed1b" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 37067 } } peers { permanent_uuid: "386df41d14874f7dbcf5d9e2044b7a04" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 38559 } }
I20260807 18:37:18.634603 32156 leader_election.cc:288] T add1286a898f4f75b7d534e37181cb86 P 4d5ced83b17f41d6958772a8c45eed1b [CANDIDATE]: Term 1 election: Requested vote from peers ddf7999748f64377a08d3d72110c2cec (127.11.14.65:44677), 386df41d14874f7dbcf5d9e2044b7a04 (127.11.14.66:38559)
I20260807 18:37:18.634918 31792 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "add1286a898f4f75b7d534e37181cb86" candidate_uuid: "4d5ced83b17f41d6958772a8c45eed1b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ddf7999748f64377a08d3d72110c2cec"
I20260807 18:37:18.634987 31933 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "add1286a898f4f75b7d534e37181cb86" candidate_uuid: "4d5ced83b17f41d6958772a8c45eed1b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "386df41d14874f7dbcf5d9e2044b7a04"
I20260807 18:37:18.635032 31792 raft_consensus.cc:3037] T add1286a898f4f75b7d534e37181cb86 P ddf7999748f64377a08d3d72110c2cec [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:37:18.635130 31933 raft_consensus.cc:3037] T add1286a898f4f75b7d534e37181cb86 P 386df41d14874f7dbcf5d9e2044b7a04 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:37:18.636235 31792 raft_consensus.cc:2454] T add1286a898f4f75b7d534e37181cb86 P ddf7999748f64377a08d3d72110c2cec [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4d5ced83b17f41d6958772a8c45eed1b in term 1.
I20260807 18:37:18.636329 31933 raft_consensus.cc:2454] T add1286a898f4f75b7d534e37181cb86 P 386df41d14874f7dbcf5d9e2044b7a04 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4d5ced83b17f41d6958772a8c45eed1b in term 1.
I20260807 18:37:18.636561 32005 leader_election.cc:302] T add1286a898f4f75b7d534e37181cb86 P 4d5ced83b17f41d6958772a8c45eed1b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4d5ced83b17f41d6958772a8c45eed1b, ddf7999748f64377a08d3d72110c2cec; no voters:
I20260807 18:37:18.636771 32156 raft_consensus.cc:2781] T add1286a898f4f75b7d534e37181cb86 P 4d5ced83b17f41d6958772a8c45eed1b [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:37:18.637048 32156 raft_consensus.cc:686] T add1286a898f4f75b7d534e37181cb86 P 4d5ced83b17f41d6958772a8c45eed1b [term 1 LEADER]: Becoming Leader. State: Replica: 4d5ced83b17f41d6958772a8c45eed1b, State: Running, Role: LEADER
I20260807 18:37:18.637425 32156 consensus_queue.cc:237] T add1286a898f4f75b7d534e37181cb86 P 4d5ced83b17f41d6958772a8c45eed1b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ddf7999748f64377a08d3d72110c2cec" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 44677 } } peers { permanent_uuid: "4d5ced83b17f41d6958772a8c45eed1b" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 37067 } } peers { permanent_uuid: "386df41d14874f7dbcf5d9e2044b7a04" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 38559 } }
I20260807 18:37:18.640731 31635 catalog_manager.cc:5495] T add1286a898f4f75b7d534e37181cb86 P 4d5ced83b17f41d6958772a8c45eed1b reported cstate change: term changed from 0 to 1, leader changed from <none> to 4d5ced83b17f41d6958772a8c45eed1b (127.11.14.67). New cstate: current_term: 1 leader_uuid: "4d5ced83b17f41d6958772a8c45eed1b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ddf7999748f64377a08d3d72110c2cec" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 44677 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4d5ced83b17f41d6958772a8c45eed1b" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 37067 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "386df41d14874f7dbcf5d9e2044b7a04" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 38559 } health_report { overall_health: UNKNOWN } } }
I20260807 18:37:18.659253 32144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:18.655588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48266 (local address 127.11.14.67:37067)
0807 18:37:18.655741 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0807 18:37:18.655746 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:37:18.655761 (+ 15us) server_negotiation.cc:407] Connection header received
0807 18:37:18.655881 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:37:18.655884 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:37:18.655942 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:37:18.656072 (+ 130us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:37:18.656334 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:18.656847 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:18.657741 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:18.657967 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:18.658078 (+ 111us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:18.658404 (+ 326us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:37:18.658827 (+ 423us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:37:18.658995 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0807 18:37:18.659082 (+ 87us) server_negotiation.cc:299] Negotiation successful
0807 18:37:18.659113 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260807 18:37:18.665238 31933 raft_consensus.cc:1261] T add1286a898f4f75b7d534e37181cb86 P 386df41d14874f7dbcf5d9e2044b7a04 [term 1 FOLLOWER]: Refusing update from remote peer 4d5ced83b17f41d6958772a8c45eed1b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:37:18.665328 31792 raft_consensus.cc:1261] T add1286a898f4f75b7d534e37181cb86 P ddf7999748f64377a08d3d72110c2cec [term 1 FOLLOWER]: Refusing update from remote peer 4d5ced83b17f41d6958772a8c45eed1b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:37:18.665988 32156 consensus_queue.cc:1035] T add1286a898f4f75b7d534e37181cb86 P 4d5ced83b17f41d6958772a8c45eed1b [LEADER]: Connected to new peer: Peer: permanent_uuid: "386df41d14874f7dbcf5d9e2044b7a04" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 38559 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:37:18.666239 32158 consensus_queue.cc:1035] T add1286a898f4f75b7d534e37181cb86 P 4d5ced83b17f41d6958772a8c45eed1b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ddf7999748f64377a08d3d72110c2cec" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 44677 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:37:18.675251 32161 mvcc.cc:205] Tried to move back new op lower bound from 7315979627165462528 to 7315979627061563392. Current Snapshot: MvccSnapshot[applied={T|T < 7315979627165462528}]
I20260807 18:37:18.683967 31635 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38392:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:37:18.684160 31635 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38392:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:37:18.686661 31635 catalog_manager.cc:5782] T 00000000000000000000000000000000 P cb005dd10bc14bcfabc73441ed20be9d: Sending DeleteTablet for 3 replicas of tablet add1286a898f4f75b7d534e37181cb86
I20260807 18:37:18.687517 31770 tablet_service.cc:1514] Processing DeleteTablet for tablet add1286a898f4f75b7d534e37181cb86 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:37:18 UTC) from {username='slave'} at 127.0.0.1:60904
I20260807 18:37:18.687618 32054 tablet_service.cc:1514] Processing DeleteTablet for tablet add1286a898f4f75b7d534e37181cb86 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:37:18 UTC) from {username='slave'} at 127.0.0.1:48262
I20260807 18:37:18.687677 31911 tablet_service.cc:1514] Processing DeleteTablet for tablet add1286a898f4f75b7d534e37181cb86 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:37:18 UTC) from {username='slave'} at 127.0.0.1:55596
I20260807 18:37:18.688091 32174 tablet_replica.cc:330] T add1286a898f4f75b7d534e37181cb86 P 386df41d14874f7dbcf5d9e2044b7a04: stopping tablet replica
I20260807 18:37:18.688318 32173 tablet_replica.cc:330] T add1286a898f4f75b7d534e37181cb86 P 4d5ced83b17f41d6958772a8c45eed1b: stopping tablet replica
I20260807 18:37:18.688463 32174 raft_consensus.cc:2229] T add1286a898f4f75b7d534e37181cb86 P 386df41d14874f7dbcf5d9e2044b7a04 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:37:18.688639 11321 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkz56XN/build/debug/bin/kudu with pid 31702
I20260807 18:37:18.688746 32173 raft_consensus.cc:2229] T add1286a898f4f75b7d534e37181cb86 P 4d5ced83b17f41d6958772a8c45eed1b [term 1 LEADER]: Raft consensus shutting down.
I20260807 18:37:18.689096 32173 raft_consensus.cc:2258] T add1286a898f4f75b7d534e37181cb86 P 4d5ced83b17f41d6958772a8c45eed1b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:37:18.689940 32174 raft_consensus.cc:2258] T add1286a898f4f75b7d534e37181cb86 P 386df41d14874f7dbcf5d9e2044b7a04 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:37:18.692122 32174 ts_tablet_manager.cc:1905] T add1286a898f4f75b7d534e37181cb86 P 386df41d14874f7dbcf5d9e2044b7a04: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:37:18.693230 32173 ts_tablet_manager.cc:1905] T add1286a898f4f75b7d534e37181cb86 P 4d5ced83b17f41d6958772a8c45eed1b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:37:18.695875 32174 ts_tablet_manager.cc:1918] T add1286a898f4f75b7d534e37181cb86 P 386df41d14874f7dbcf5d9e2044b7a04: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:37:18.696000 32173 ts_tablet_manager.cc:1918] T add1286a898f4f75b7d534e37181cb86 P 4d5ced83b17f41d6958772a8c45eed1b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:37:18.696099 32173 log.cc:1196] T add1286a898f4f75b7d534e37181cb86 P 4d5ced83b17f41d6958772a8c45eed1b: Deleting WAL directory at /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786127579900523-11321-0/minicluster-data/ts-2/wal/wals/add1286a898f4f75b7d534e37181cb86
I20260807 18:37:18.696056 32174 log.cc:1196] T add1286a898f4f75b7d534e37181cb86 P 386df41d14874f7dbcf5d9e2044b7a04: Deleting WAL directory at /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786127579900523-11321-0/minicluster-data/ts-1/wal/wals/add1286a898f4f75b7d534e37181cb86
I20260807 18:37:18.696439 32173 ts_tablet_manager.cc:1939] T add1286a898f4f75b7d534e37181cb86 P 4d5ced83b17f41d6958772a8c45eed1b: Deleting consensus metadata
I20260807 18:37:18.696602 32174 ts_tablet_manager.cc:1939] T add1286a898f4f75b7d534e37181cb86 P 386df41d14874f7dbcf5d9e2044b7a04: Deleting consensus metadata
I20260807 18:37:18.697674 31620 catalog_manager.cc:4841] TS 4d5ced83b17f41d6958772a8c45eed1b (127.11.14.67:37067): tablet add1286a898f4f75b7d534e37181cb86 (table test-table [id=0bb083847c8649d49a7addac1d37eeca]) successfully deleted
I20260807 18:37:18.697947 31619 catalog_manager.cc:4841] TS 386df41d14874f7dbcf5d9e2044b7a04 (127.11.14.66:38559): tablet add1286a898f4f75b7d534e37181cb86 (table test-table [id=0bb083847c8649d49a7addac1d37eeca]) successfully deleted
W20260807 18:37:18.699494 31619 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.14.65:44677 (error 108)
I20260807 18:37:18.699815 11321 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkz56XN/build/debug/bin/kudu with pid 31844
I20260807 18:37:18.704542 32145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:18.704133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.65:44677 (local address 127.0.0.1:60920)
0807 18:37:18.704368 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0807 18:37:18.704443 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.65:44677: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135}
W20260807 18:37:18.704816 31619 catalog_manager.cc:4584] TS ddf7999748f64377a08d3d72110c2cec (127.11.14.65:44677): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet add1286a898f4f75b7d534e37181cb86: Network error: Client connection negotiation failed: client connection to 127.11.14.65:44677: connect: Connection refused (error 111)
I20260807 18:37:18.716020 11321 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkz56XN/build/debug/bin/kudu with pid 31988
I20260807 18:37:18.724756 11321 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkz56XN/build/debug/bin/kudu with pid 31602
2026-08-07T18:37:18Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (1134 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127579900523-11321-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:37:18 dist-test-slave-6k22 krb5kdc[32178](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:37:18 dist-test-slave-6k22 krb5kdc[32178](info): set up 2 sockets
Aug 07 18:37:18 dist-test-slave-6k22 krb5kdc[32178](info): commencing operation
krb5kdc: starting...
W20260807 18:37:19.818564 11321 mini_kdc.cc:121] Time spent starting KDC: real 1.070s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127579900523-11321-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127579900523-11321-0/krb5kdc/test-user.keytab.
Aug 07 18:37:19 dist-test-slave-6k22 krb5kdc[32178](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127839, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:37:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:37:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.14.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.14.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.14.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.126.keytab.
Entry for principal kudu/127.11.14.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.126.keytab.
I20260807 18:37:19.950559 11321 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkz56XN/build/debug/bin/kudu
/tmp/dist-test-taskkz56XN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127579900523-11321-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127579900523-11321-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127579900523-11321-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127579900523-11321-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.14.126:35917
--webserver_interface=127.11.14.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.126.keytab
--principal=kudu/127.11.14.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.14.84:46193
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.14.126:35917
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127579900523-11321-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127579900523-11321-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127579900523-11321-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:37:20.067746 32193 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:37:20.068024 32193 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:37:20.068071 32193 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:37:20.072389 32193 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:37:20.072446 32193 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:37:20.072471 32193 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:37:20.072494 32193 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:37:20.072513 32193 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:37:20.078379 32193 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.14.84:46193
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127579900523-11321-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127579900523-11321-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.14.126:35917
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.126.keytab
--principal=kudu/127.11.14.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.14.126:35917
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127579900523-11321-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.14.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32193
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127579900523-11321-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:37:20.079494 32193 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:37:20.080612 32193 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:37:20.090193 32199 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:37:20.090381 32193 server_base.cc:1034] running on GCE node
W20260807 18:37:20.090184 32202 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:37:20.090184 32198 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:37:20.091104 32193 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:37:20.092324 32193 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:37:20.093525 32193 hybrid_clock.cc:648] HybridClock initialized: now 1786127840093514 us; error 25 us; skew 500 ppm
Aug 07 18:37:20 dist-test-slave-6k22 krb5kdc[32178](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127840, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:37:20.096374 32193 init.cc:377] Logged in from keytab as kudu/127.11.14.126@KRBTEST.COM (short username kudu)
I20260807 18:37:20.097234 32193 webserver.cc:460] Webserver started at http://127.11.14.126:41335/ using document root <none> and password file <none>
I20260807 18:37:20.097800 32193 fs_manager.cc:359] Metadata directory not provided
I20260807 18:37:20.097890 32193 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:37:20.098130 32193 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:37:20.100268 32193 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127579900523-11321-0/minicluster-data/master-0/data/instance:
uuid: "efd7333152f742efb3ed32d10d6539b2"
format_stamp: "Formatted at 2026-08-07 18:37:20 on dist-test-slave-6k22"
server_key: "e8a96be0894f3e3fecd346bad5727e14"
server_key_iv: "f56c6b8996ea6b741b829b3d0e069d9d"
server_key_version: "encryptionkey@0"
I20260807 18:37:20.100834 32193 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127579900523-11321-0/minicluster-data/master-0/wal/instance:
uuid: "efd7333152f742efb3ed32d10d6539b2"
format_stamp: "Formatted at 2026-08-07 18:37:20 on dist-test-slave-6k22"
server_key: "e8a96be0894f3e3fecd346bad5727e14"
server_key_iv: "f56c6b8996ea6b741b829b3d0e069d9d"
server_key_version: "encryptionkey@0"
I20260807 18:37:20.104494 32193 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 18:37:20.107182 32210 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:37:20.108449 32193 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 18:37:20.108589 32193 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127579900523-11321-0/minicluster-data/master-0/data,/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127579900523-11321-0/minicluster-data/master-0/wal
uuid: "efd7333152f742efb3ed32d10d6539b2"
format_stamp: "Formatted at 2026-08-07 18:37:20 on dist-test-slave-6k22"
server_key: "e8a96be0894f3e3fecd346bad5727e14"
server_key_iv: "f56c6b8996ea6b741b829b3d0e069d9d"
server_key_version: "encryptionkey@0"
I20260807 18:37:20.108700 32193 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127579900523-11321-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127579900523-11321-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127579900523-11321-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:37:20.128160 32193 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:37:20.130900 32193 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:37:20.131102 32193 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:37:20.138325 32193 rpc_server.cc:307] RPC server started. Bound to: 127.11.14.126:35917
I20260807 18:37:20.138355 32266 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.14.126:35917 every 8 connection(s)
I20260807 18:37:20.139285 32193 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127579900523-11321-0/minicluster-data/master-0/data/info.pb
I20260807 18:37:20.142217 32268 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:37:20.147931 11321 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkz56XN/build/debug/bin/kudu as pid 32193
I20260807 18:37:20.148036 32268 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P efd7333152f742efb3ed32d10d6539b2: Bootstrap starting.
I20260807 18:37:20.148074 11321 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127579900523-11321-0/minicluster-data/master-0/wal/instance
I20260807 18:37:20.148388 11321 external_mini_cluster.cc:1428] Setting key c28341caa3651415c6f96c90ff58543e
I20260807 18:37:20.150799 32268 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P efd7333152f742efb3ed32d10d6539b2: Neither blocks nor log segments found. Creating new log.
I20260807 18:37:20.151549 32268 log.cc:824] T 00000000000000000000000000000000 P efd7333152f742efb3ed32d10d6539b2: Log is configured to *not* fsync() on all Append() calls
I20260807 18:37:20.153501 32268 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P efd7333152f742efb3ed32d10d6539b2: No bootstrap required, opened a new log
Aug 07 18:37:20 dist-test-slave-6k22 krb5kdc[32178](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127839, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.14.126@KRBTEST.COM
I20260807 18:37:20.158021 32268 raft_consensus.cc:348] T 00000000000000000000000000000000 P efd7333152f742efb3ed32d10d6539b2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "efd7333152f742efb3ed32d10d6539b2" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 35917 } }
I20260807 18:37:20.158264 32268 raft_consensus.cc:374] T 00000000000000000000000000000000 P efd7333152f742efb3ed32d10d6539b2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:37:20.158366 32268 raft_consensus.cc:729] T 00000000000000000000000000000000 P efd7333152f742efb3ed32d10d6539b2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efd7333152f742efb3ed32d10d6539b2, State: Initialized, Role: FOLLOWER
I20260807 18:37:20.158854 32268 consensus_queue.cc:260] T 00000000000000000000000000000000 P efd7333152f742efb3ed32d10d6539b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "efd7333152f742efb3ed32d10d6539b2" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 35917 } }
I20260807 18:37:20.159010 32268 raft_consensus.cc:388] T 00000000000000000000000000000000 P efd7333152f742efb3ed32d10d6539b2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:37:20.159093 32268 raft_consensus.cc:482] T 00000000000000000000000000000000 P efd7333152f742efb3ed32d10d6539b2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:37:20.159211 32268 raft_consensus.cc:3037] T 00000000000000000000000000000000 P efd7333152f742efb3ed32d10d6539b2 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:37:20.160414 32268 raft_consensus.cc:504] T 00000000000000000000000000000000 P efd7333152f742efb3ed32d10d6539b2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "efd7333152f742efb3ed32d10d6539b2" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 35917 } }
I20260807 18:37:20.160797 32268 leader_election.cc:302] T 00000000000000000000000000000000 P efd7333152f742efb3ed32d10d6539b2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: efd7333152f742efb3ed32d10d6539b2; no voters:
I20260807 18:37:20.161149 32268 leader_election.cc:288] T 00000000000000000000000000000000 P efd7333152f742efb3ed32d10d6539b2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:37:20.161291 32275 raft_consensus.cc:2781] T 00000000000000000000000000000000 P efd7333152f742efb3ed32d10d6539b2 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:37:20.161527 32275 raft_consensus.cc:686] T 00000000000000000000000000000000 P efd7333152f742efb3ed32d10d6539b2 [term 1 LEADER]: Becoming Leader. State: Replica: efd7333152f742efb3ed32d10d6539b2, State: Running, Role: LEADER
I20260807 18:37:20.161921 32275 consensus_queue.cc:237] T 00000000000000000000000000000000 P efd7333152f742efb3ed32d10d6539b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "efd7333152f742efb3ed32d10d6539b2" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 35917 } }
I20260807 18:37:20.162534 32268 sys_catalog.cc:564] T 00000000000000000000000000000000 P efd7333152f742efb3ed32d10d6539b2 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:37:20.164115 32271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:20.149889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56600 (local address 127.11.14.126:35917)
0807 18:37:20.150324 (+ 435us) server_negotiation.cc:206] Beginning negotiation
0807 18:37:20.150339 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 18:37:20.150376 (+ 37us) server_negotiation.cc:407] Connection header received
0807 18:37:20.151120 (+ 744us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:37:20.151141 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:37:20.151471 (+ 330us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:37:20.151882 (+ 411us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:37:20.152736 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:20.153764 (+ 1028us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:20.154631 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:20.155105 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:20.155350 (+ 245us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:20.158399 (+ 3049us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:37:20.158406 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:37:20.158425 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:37:20.158460 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:37:20.160585 (+ 2125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:37:20.161023 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:37:20.161030 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:37:20.161037 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:37:20.161192 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:37:20.161568 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:37:20.161572 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:37:20.161574 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:37:20.162093 (+ 519us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:37:20.162314 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0807 18:37:20.162648 (+ 334us) server_negotiation.cc:299] Negotiation successful
0807 18:37:20.162997 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":294,"thread_start_us":123,"threads_started":1}
I20260807 18:37:20.165630 32277 sys_catalog.cc:455] T 00000000000000000000000000000000 P efd7333152f742efb3ed32d10d6539b2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader efd7333152f742efb3ed32d10d6539b2. Latest consensus state: current_term: 1 leader_uuid: "efd7333152f742efb3ed32d10d6539b2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "efd7333152f742efb3ed32d10d6539b2" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 35917 } } }
I20260807 18:37:20.165639 32276 sys_catalog.cc:455] T 00000000000000000000000000000000 P efd7333152f742efb3ed32d10d6539b2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "efd7333152f742efb3ed32d10d6539b2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "efd7333152f742efb3ed32d10d6539b2" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 35917 } } }
I20260807 18:37:20.165872 32277 sys_catalog.cc:458] T 00000000000000000000000000000000 P efd7333152f742efb3ed32d10d6539b2 [sys.catalog]: This master's current role is: LEADER
I20260807 18:37:20.165874 32276 sys_catalog.cc:458] T 00000000000000000000000000000000 P efd7333152f742efb3ed32d10d6539b2 [sys.catalog]: This master's current role is: LEADER
W20260807 18:37:20.168510 32292 catalog_manager.cc:1558] T 00000000000000000000000000000000 P efd7333152f742efb3ed32d10d6539b2: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 18:37:20.168577 32292 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 18:37:20.168659 32295 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:37:20.169401 32295 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:37:20.175024 32295 catalog_manager.cc:1347] Generated new cluster ID: 6017e80002e04ef5a5b2f08b2fb6d821
I20260807 18:37:20.175148 32295 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:37:20.219429 32295 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:37:20.220458 32295 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:37:20.228514 32295 catalog_manager.cc:5868] T 00000000000000000000000000000000 P efd7333152f742efb3ed32d10d6539b2: Generated new TSK 0
I20260807 18:37:20.229233 32295 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.14.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.14.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.14.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.65.keytab.
Entry for principal kudu/127.11.14.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.65.keytab.
I20260807 18:37:20.266520 11321 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkz56XN/build/debug/bin/kudu
/tmp/dist-test-taskkz56XN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127579900523-11321-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127579900523-11321-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127579900523-11321-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127579900523-11321-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.14.65:0
--local_ip_for_outbound_sockets=127.11.14.65
--webserver_interface=127.11.14.65
--webserver_port=0
--tserver_master_addrs=127.11.14.126:35917
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.65.keytab
--principal=kudu/127.11.14.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.14.84:46193
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127579900523-11321-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127579900523-11321-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127579900523-11321-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:37:20.381120 32301 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:37:20.381387 32301 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:37:20.381430 32301 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:37:20.385537 32301 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:37:20.385603 32301 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:37:20.385661 32301 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.14.65
I20260807 18:37:20.391348 32301 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.14.84:46193
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127579900523-11321-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127579900523-11321-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.65.keytab
--principal=kudu/127.11.14.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.14.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127579900523-11321-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.14.65
--webserver_port=0
--tserver_master_addrs=127.11.14.126:35917
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32301
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.14.65
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127579900523-11321-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:37:20.392438 32301 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:37:20.393482 32301 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:37:20.403059 32311 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:37:20.403082 32308 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:37:20.403076 32309 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:37:20.403118 32301 server_base.cc:1034] running on GCE node
I20260807 18:37:20.403718 32301 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:37:20.404377 32301 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:37:20.405561 32301 hybrid_clock.cc:648] HybridClock initialized: now 1786127840405552 us; error 31 us; skew 500 ppm
Aug 07 18:37:20 dist-test-slave-6k22 krb5kdc[32178](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127840, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:37:20.408259 32301 init.cc:377] Logged in from keytab as kudu/127.11.14.65@KRBTEST.COM (short username kudu)
I20260807 18:37:20.409111 32301 webserver.cc:460] Webserver started at http://127.11.14.65:39907/ using document root <none> and password file <none>
I20260807 18:37:20.409621 32301 fs_manager.cc:359] Metadata directory not provided
I20260807 18:37:20.409677 32301 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:37:20.409853 32301 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:37:20.411950 32301 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127579900523-11321-0/minicluster-data/ts-0/data/instance:
uuid: "7baa370b5aad4902a6e7cb2ac21d8ab7"
format_stamp: "Formatted at 2026-08-07 18:37:20 on dist-test-slave-6k22"
server_key: "8d8c14dbd989f44001230bac8f9e859c"
server_key_iv: "19e28f938a8cd0561e0c0bbd15a2699e"
server_key_version: "encryptionkey@0"
I20260807 18:37:20.412463 32301 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127579900523-11321-0/minicluster-data/ts-0/wal/instance:
uuid: "7baa370b5aad4902a6e7cb2ac21d8ab7"
format_stamp: "Formatted at 2026-08-07 18:37:20 on dist-test-slave-6k22"
server_key: "8d8c14dbd989f44001230bac8f9e859c"
server_key_iv: "19e28f938a8cd0561e0c0bbd15a2699e"
server_key_version: "encryptionkey@0"
I20260807 18:37:20.416249 32301 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.002s
I20260807 18:37:20.418653 32320 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:37:20.419976 32301 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260807 18:37:20.420105 32301 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127579900523-11321-0/minicluster-data/ts-0/data,/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127579900523-11321-0/minicluster-data/ts-0/wal
uuid: "7baa370b5aad4902a6e7cb2ac21d8ab7"
format_stamp: "Formatted at 2026-08-07 18:37:20 on dist-test-slave-6k22"
server_key: "8d8c14dbd989f44001230bac8f9e859c"
server_key_iv: "19e28f938a8cd0561e0c0bbd15a2699e"
server_key_version: "encryptionkey@0"
I20260807 18:37:20.420213 32301 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127579900523-11321-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127579900523-11321-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127579900523-11321-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:37:20.432224 32301 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:37:20.435180 32301 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:37:20.435397 32301 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:37:20.436080 32301 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:37:20.436920 32301 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:37:20.436966 32301 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:37:20.437036 32301 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:37:20.437067 32301 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:37:20.445976 32301 rpc_server.cc:307] RPC server started. Bound to: 127.11.14.65:44953
I20260807 18:37:20.446010 32446 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.14.65:44953 every 8 connection(s)
I20260807 18:37:20.446875 32301 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127579900523-11321-0/minicluster-data/ts-0/data/info.pb
I20260807 18:37:20.453732 11321 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkz56XN/build/debug/bin/kudu as pid 32301
I20260807 18:37:20.453876 11321 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127579900523-11321-0/minicluster-data/ts-0/wal/instance
I20260807 18:37:20.454173 11321 external_mini_cluster.cc:1428] Setting key a7a63ef1f3a3de6a2b092186a5b4afb6
Aug 07 18:37:20 dist-test-slave-6k22 krb5kdc[32178](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127840, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.65@KRBTEST.COM for kudu/127.11.14.126@KRBTEST.COM
I20260807 18:37:20.461695 32271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:20.448731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.65:56727 (local address 127.11.14.126:35917)
0807 18:37:20.448900 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0807 18:37:20.448904 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:37:20.449685 (+ 781us) server_negotiation.cc:407] Connection header received
0807 18:37:20.450987 (+ 1302us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:37:20.450991 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:37:20.451053 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:37:20.451162 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:37:20.452409 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:20.452947 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:20.454512 (+ 1565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:20.454847 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:20.454933 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:20.457733 (+ 2800us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:37:20.457738 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:37:20.457743 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:37:20.457779 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:37:20.459521 (+ 1742us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:37:20.460036 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:37:20.460042 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:37:20.460047 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:37:20.460145 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:37:20.460486 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:37:20.460492 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:37:20.460497 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:37:20.460780 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:37:20.460894 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0807 18:37:20.461358 (+ 464us) server_negotiation.cc:299] Negotiation successful
0807 18:37:20.461491 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260807 18:37:20.462492 32450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:20.449084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:35917 (local address 127.11.14.65:56727)
0807 18:37:20.449531 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0807 18:37:20.449575 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 18:37:20.450753 (+ 1178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:37:20.451348 (+ 595us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:37:20.451355 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:37:20.451781 (+ 426us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:37:20.452259 (+ 478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:37:20.452273 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:37:20.453078 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:37:20.453088 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:37:20.454364 (+ 1276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:37:20.454376 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:37:20.455011 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:37:20.455015 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:37:20.455164 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:20.455880 (+ 716us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:37:20.455901 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:37:20.457560 (+ 1659us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:37:20.459662 (+ 2102us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:37:20.459666 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:37:20.459677 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:37:20.459931 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:37:20.460241 (+ 310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:37:20.460243 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:37:20.460245 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:37:20.460384 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:37:20.460888 (+ 504us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:37:20.460893 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:37:20.461109 (+ 216us) client_negotiation.cc:769] Sending connection context
0807 18:37:20.461261 (+ 152us) client_negotiation.cc:240] Negotiation successful
0807 18:37:20.461516 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":123,"threads_started":1}
I20260807 18:37:20.463886 32447 heartbeater.cc:347] Connected to a master server at 127.11.14.126:35917
I20260807 18:37:20.464171 32447 heartbeater.cc:464] Registering TS with master...
I20260807 18:37:20.464846 32447 heartbeater.cc:511] Master 127.11.14.126:35917 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.11.14.66@KRBTEST.COM; defaulting to no policy
I20260807 18:37:20.466591 32228 ts_manager.cc:194] Registered new tserver with Master: 7baa370b5aad4902a6e7cb2ac21d8ab7 (127.11.14.65:44953)
I20260807 18:37:20.467821 32228 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.14.65@KRBTEST.COM'} at 127.11.14.65:56727
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.14.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.14.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.66.keytab.
Entry for principal kudu/127.11.14.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.66.keytab.
I20260807 18:37:20.484822 11321 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkz56XN/build/debug/bin/kudu
/tmp/dist-test-taskkz56XN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127579900523-11321-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127579900523-11321-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127579900523-11321-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127579900523-11321-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.14.66:0
--local_ip_for_outbound_sockets=127.11.14.66
--webserver_interface=127.11.14.66
--webserver_port=0
--tserver_master_addrs=127.11.14.126:35917
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.66.keytab
--principal=kudu/127.11.14.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.14.84:46193
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127579900523-11321-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127579900523-11321-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127579900523-11321-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:37:20.599882 32453 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:37:20.600170 32453 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:37:20.600247 32453 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:37:20.604234 32453 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:37:20.604336 32453 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:37:20.604426 32453 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.14.66
I20260807 18:37:20.609753 32453 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.14.84:46193
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127579900523-11321-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127579900523-11321-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.66.keytab
--principal=kudu/127.11.14.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.14.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127579900523-11321-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.14.66
--webserver_port=0
--tserver_master_addrs=127.11.14.126:35917
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32453
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.14.66
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127579900523-11321-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:37:20.610944 32453 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:37:20.612262 32453 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:37:20.622398 32460 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:37:20.622424 32459 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:37:20.622398 32462 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:37:20.622815 32453 server_base.cc:1034] running on GCE node
I20260807 18:37:20.623242 32453 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:37:20.624034 32453 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:37:20.625216 32453 hybrid_clock.cc:648] HybridClock initialized: now 1786127840625197 us; error 28 us; skew 500 ppm
Aug 07 18:37:20 dist-test-slave-6k22 krb5kdc[32178](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127840, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:37:20.628201 32453 init.cc:377] Logged in from keytab as kudu/127.11.14.66@KRBTEST.COM (short username kudu)
I20260807 18:37:20.629079 32453 webserver.cc:460] Webserver started at http://127.11.14.66:40889/ using document root <none> and password file <none>
I20260807 18:37:20.629657 32453 fs_manager.cc:359] Metadata directory not provided
I20260807 18:37:20.629719 32453 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:37:20.629971 32453 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:37:20.632114 32453 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127579900523-11321-0/minicluster-data/ts-1/data/instance:
uuid: "5c53f91c64bc4245a7320bad5c78ed76"
format_stamp: "Formatted at 2026-08-07 18:37:20 on dist-test-slave-6k22"
server_key: "223ba3f27cd8e3a8492f2deca96caaca"
server_key_iv: "1594ab75efd185314f35ef882fd6dee0"
server_key_version: "encryptionkey@0"
I20260807 18:37:20.632689 32453 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127579900523-11321-0/minicluster-data/ts-1/wal/instance:
uuid: "5c53f91c64bc4245a7320bad5c78ed76"
format_stamp: "Formatted at 2026-08-07 18:37:20 on dist-test-slave-6k22"
server_key: "223ba3f27cd8e3a8492f2deca96caaca"
server_key_iv: "1594ab75efd185314f35ef882fd6dee0"
server_key_version: "encryptionkey@0"
I20260807 18:37:20.636551 32453 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260807 18:37:20.638962 32469 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:37:20.640247 32453 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 18:37:20.640386 32453 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127579900523-11321-0/minicluster-data/ts-1/data,/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127579900523-11321-0/minicluster-data/ts-1/wal
uuid: "5c53f91c64bc4245a7320bad5c78ed76"
format_stamp: "Formatted at 2026-08-07 18:37:20 on dist-test-slave-6k22"
server_key: "223ba3f27cd8e3a8492f2deca96caaca"
server_key_iv: "1594ab75efd185314f35ef882fd6dee0"
server_key_version: "encryptionkey@0"
I20260807 18:37:20.640503 32453 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127579900523-11321-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127579900523-11321-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127579900523-11321-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:37:20.652266 32453 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:37:20.655732 32453 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:37:20.655949 32453 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:37:20.656567 32453 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:37:20.657439 32453 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:37:20.657521 32453 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:37:20.657611 32453 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:37:20.657663 32453 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:37:20.667567 32453 rpc_server.cc:307] RPC server started. Bound to: 127.11.14.66:45307
I20260807 18:37:20.667614 32588 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.14.66:45307 every 8 connection(s)
I20260807 18:37:20.668535 32453 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127579900523-11321-0/minicluster-data/ts-1/data/info.pb
I20260807 18:37:20.672574 11321 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkz56XN/build/debug/bin/kudu as pid 32453
I20260807 18:37:20.672670 11321 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127579900523-11321-0/minicluster-data/ts-1/wal/instance
I20260807 18:37:20.673026 11321 external_mini_cluster.cc:1428] Setting key 081189d856f2c982630507c6834680e0
Aug 07 18:37:20 dist-test-slave-6k22 krb5kdc[32178](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127840, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.66@KRBTEST.COM for kudu/127.11.14.126@KRBTEST.COM
I20260807 18:37:20.684367 32271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:20.670382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.66:42739 (local address 127.11.14.126:35917)
0807 18:37:20.670518 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0807 18:37:20.670522 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:37:20.671430 (+ 908us) server_negotiation.cc:407] Connection header received
0807 18:37:20.672952 (+ 1522us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:37:20.672956 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:37:20.673018 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:37:20.673121 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:37:20.674525 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:20.675265 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:20.677058 (+ 1793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:20.677296 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:20.677408 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:20.680554 (+ 3146us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:37:20.680568 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:37:20.680570 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:37:20.680601 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:37:20.682096 (+ 1495us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:37:20.682630 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:37:20.682634 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:37:20.682635 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:37:20.682716 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:37:20.683074 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:37:20.683077 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:37:20.683079 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:37:20.683264 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:37:20.683375 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0807 18:37:20.684029 (+ 654us) server_negotiation.cc:299] Negotiation successful
0807 18:37:20.684165 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
WARNING: no policy specified for kudu/127.11.14.67@KRBTEST.COM; defaulting to no policy
I20260807 18:37:20.685257 32591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:20.670810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:35917 (local address 127.11.14.66:42739)
0807 18:37:20.671270 (+ 460us) negotiation.cc:107] Waiting for socket to connect
0807 18:37:20.671314 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 18:37:20.672695 (+ 1381us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:37:20.673316 (+ 621us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:37:20.673323 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:37:20.673812 (+ 489us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:37:20.674353 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:37:20.674375 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:37:20.675412 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:37:20.675415 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:37:20.676904 (+ 1489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:37:20.676917 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:37:20.677475 (+ 558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:37:20.677483 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:37:20.677720 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:20.678570 (+ 850us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:37:20.678597 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:37:20.680372 (+ 1775us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:37:20.682221 (+ 1849us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:37:20.682227 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:37:20.682244 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:37:20.682517 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:37:20.682812 (+ 295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:37:20.682815 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:37:20.682817 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:37:20.682969 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:37:20.683389 (+ 420us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:37:20.683397 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:37:20.683735 (+ 338us) client_negotiation.cc:769] Sending connection context
0807 18:37:20.683933 (+ 198us) client_negotiation.cc:240] Negotiation successful
0807 18:37:20.684302 (+ 369us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":120,"threads_started":1}
I20260807 18:37:20.686524 32589 heartbeater.cc:347] Connected to a master server at 127.11.14.126:35917
I20260807 18:37:20.686836 32589 heartbeater.cc:464] Registering TS with master...
I20260807 18:37:20.687399 32589 heartbeater.cc:511] Master 127.11.14.126:35917 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.14.67@KRBTEST.COM" created.
I20260807 18:37:20.688702 32228 ts_manager.cc:194] Registered new tserver with Master: 5c53f91c64bc4245a7320bad5c78ed76 (127.11.14.66:45307)
I20260807 18:37:20.689574 32228 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.14.66@KRBTEST.COM'} at 127.11.14.66:42739
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.14.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.67.keytab.
Entry for principal kudu/127.11.14.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.67.keytab.
I20260807 18:37:20.706509 11321 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkz56XN/build/debug/bin/kudu
/tmp/dist-test-taskkz56XN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127579900523-11321-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127579900523-11321-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127579900523-11321-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127579900523-11321-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.14.67:0
--local_ip_for_outbound_sockets=127.11.14.67
--webserver_interface=127.11.14.67
--webserver_port=0
--tserver_master_addrs=127.11.14.126:35917
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.67.keytab
--principal=kudu/127.11.14.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.14.84:46193
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127579900523-11321-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127579900523-11321-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127579900523-11321-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:37:20.829838 32595 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:37:20.830124 32595 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:37:20.830174 32595 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:37:20.834684 32595 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:37:20.834887 32595 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:37:20.834992 32595 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.14.67
I20260807 18:37:20.841099 32595 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.14.84:46193
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127579900523-11321-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127579900523-11321-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.67.keytab
--principal=kudu/127.11.14.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.14.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127579900523-11321-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.14.67
--webserver_port=0
--tserver_master_addrs=127.11.14.126:35917
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32595
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.14.67
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127579900523-11321-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:37:20.842244 32595 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:37:20.843308 32595 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:37:20.854060 32604 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:37:20.854040 32602 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:37:20.854059 32601 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:37:20.854291 32595 server_base.cc:1034] running on GCE node
I20260807 18:37:20.854800 32595 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:37:20.855458 32595 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:37:20.856639 32595 hybrid_clock.cc:648] HybridClock initialized: now 1786127840856616 us; error 33 us; skew 500 ppm
Aug 07 18:37:20 dist-test-slave-6k22 krb5kdc[32178](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127840, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:37:20.859380 32595 init.cc:377] Logged in from keytab as kudu/127.11.14.67@KRBTEST.COM (short username kudu)
I20260807 18:37:20.860366 32595 webserver.cc:460] Webserver started at http://127.11.14.67:36461/ using document root <none> and password file <none>
I20260807 18:37:20.860972 32595 fs_manager.cc:359] Metadata directory not provided
I20260807 18:37:20.861068 32595 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:37:20.861294 32595 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:37:20.863611 32595 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127579900523-11321-0/minicluster-data/ts-2/data/instance:
uuid: "5b957e65b83347f3a1f43b8005355979"
format_stamp: "Formatted at 2026-08-07 18:37:20 on dist-test-slave-6k22"
server_key: "c87c7827b2067d566aba9437c50ea017"
server_key_iv: "3d9e7f3a10c6e1cb5d67c462ddfb3db1"
server_key_version: "encryptionkey@0"
I20260807 18:37:20.864215 32595 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127579900523-11321-0/minicluster-data/ts-2/wal/instance:
uuid: "5b957e65b83347f3a1f43b8005355979"
format_stamp: "Formatted at 2026-08-07 18:37:20 on dist-test-slave-6k22"
server_key: "c87c7827b2067d566aba9437c50ea017"
server_key_iv: "3d9e7f3a10c6e1cb5d67c462ddfb3db1"
server_key_version: "encryptionkey@0"
I20260807 18:37:20.868026 32595 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260807 18:37:20.870460 32613 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:37:20.871752 32595 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 18:37:20.871901 32595 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127579900523-11321-0/minicluster-data/ts-2/data,/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127579900523-11321-0/minicluster-data/ts-2/wal
uuid: "5b957e65b83347f3a1f43b8005355979"
format_stamp: "Formatted at 2026-08-07 18:37:20 on dist-test-slave-6k22"
server_key: "c87c7827b2067d566aba9437c50ea017"
server_key_iv: "3d9e7f3a10c6e1cb5d67c462ddfb3db1"
server_key_version: "encryptionkey@0"
I20260807 18:37:20.872017 32595 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127579900523-11321-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127579900523-11321-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127579900523-11321-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:37:20.885273 32595 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:37:20.888294 32595 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:37:20.888495 32595 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:37:20.889114 32595 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:37:20.889966 32595 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:37:20.890036 32595 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:37:20.890110 32595 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:37:20.890161 32595 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:37:20.899909 32595 rpc_server.cc:307] RPC server started. Bound to: 127.11.14.67:36863
I20260807 18:37:20.899936 32737 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.14.67:36863 every 8 connection(s)
I20260807 18:37:20.900830 32595 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127579900523-11321-0/minicluster-data/ts-2/data/info.pb
I20260807 18:37:20.904129 11321 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkz56XN/build/debug/bin/kudu as pid 32595
I20260807 18:37:20.904258 11321 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127579900523-11321-0/minicluster-data/ts-2/wal/instance
I20260807 18:37:20.904544 11321 external_mini_cluster.cc:1428] Setting key e256520d982c577c4090be1def248a3d
Aug 07 18:37:20 dist-test-slave-6k22 krb5kdc[32178](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127840, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.67@KRBTEST.COM for kudu/127.11.14.126@KRBTEST.COM
I20260807 18:37:20.914618 32271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:20.902626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.67:54981 (local address 127.11.14.126:35917)
0807 18:37:20.902796 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0807 18:37:20.902801 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:37:20.903659 (+ 858us) server_negotiation.cc:407] Connection header received
0807 18:37:20.905257 (+ 1598us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:37:20.905261 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:37:20.905332 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:37:20.905447 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:37:20.906650 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:20.907177 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:20.908387 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:20.908607 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:20.908714 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:20.911099 (+ 2385us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:37:20.911108 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:37:20.911110 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:37:20.911135 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:37:20.912491 (+ 1356us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:37:20.913061 (+ 570us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:37:20.913069 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:37:20.913070 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:37:20.913145 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:37:20.913491 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:37:20.913495 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:37:20.913497 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:37:20.913675 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:37:20.913781 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0807 18:37:20.914304 (+ 523us) server_negotiation.cc:299] Negotiation successful
0807 18:37:20.914444 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260807 18:37:20.915637 32742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:20.903028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:35917 (local address 127.11.14.67:54981)
0807 18:37:20.903477 (+ 449us) negotiation.cc:107] Waiting for socket to connect
0807 18:37:20.903519 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 18:37:20.904854 (+ 1335us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:37:20.905635 (+ 781us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:37:20.905641 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:37:20.906017 (+ 376us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:37:20.906506 (+ 489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:37:20.906519 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:37:20.907308 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:37:20.907313 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:37:20.908257 (+ 944us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:37:20.908268 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:37:20.908768 (+ 500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:37:20.908782 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:37:20.908950 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:20.909503 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:37:20.909520 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:37:20.910934 (+ 1414us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:37:20.912618 (+ 1684us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:37:20.912623 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:37:20.912639 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:37:20.912927 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:37:20.913239 (+ 312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:37:20.913242 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:37:20.913244 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:37:20.913383 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:37:20.913805 (+ 422us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:37:20.913811 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:37:20.914052 (+ 241us) client_negotiation.cc:769] Sending connection context
0807 18:37:20.914246 (+ 194us) client_negotiation.cc:240] Negotiation successful
0807 18:37:20.914569 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":107,"threads_started":1}
I20260807 18:37:20.916816 32740 heartbeater.cc:347] Connected to a master server at 127.11.14.126:35917
I20260807 18:37:20.917083 32740 heartbeater.cc:464] Registering TS with master...
I20260807 18:37:20.917682 32740 heartbeater.cc:511] Master 127.11.14.126:35917 requested a full tablet report, sending...
I20260807 18:37:20.918744 32228 ts_manager.cc:194] Registered new tserver with Master: 5b957e65b83347f3a1f43b8005355979 (127.11.14.67:36863)
I20260807 18:37:20.919258 32228 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.14.67@KRBTEST.COM'} at 127.11.14.67:54981
I20260807 18:37:20.929949 11321 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 18:37:20.938931 32271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:20.932105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56610 (local address 127.11.14.126:35917)
0807 18:37:20.932220 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0807 18:37:20.932224 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:37:20.932426 (+ 202us) server_negotiation.cc:407] Connection header received
0807 18:37:20.932514 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:37:20.932517 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:37:20.932554 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:37:20.932619 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:37:20.933293 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:20.933784 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:20.934575 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:20.934784 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:20.934864 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:20.936286 (+ 1422us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:37:20.936289 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:37:20.936291 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:37:20.936314 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:37:20.937514 (+ 1200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:37:20.937894 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:37:20.937897 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:37:20.937898 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:37:20.937969 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:37:20.938193 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:37:20.938195 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:37:20.938196 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:37:20.938424 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:37:20.938528 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0807 18:37:20.938723 (+ 195us) server_negotiation.cc:299] Negotiation successful
0807 18:37:20.938813 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260807 18:37:20.941639 32228 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56610:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 18:37:20.944783 32228 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 18:37:20.959424 32754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:20.953406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34364 (local address 127.11.14.65:44953)
0807 18:37:20.954346 (+ 940us) server_negotiation.cc:206] Beginning negotiation
0807 18:37:20.954364 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0807 18:37:20.954393 (+ 29us) server_negotiation.cc:407] Connection header received
0807 18:37:20.954478 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:37:20.954487 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:37:20.954704 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:37:20.955252 (+ 548us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:37:20.955434 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:20.956648 (+ 1214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:20.958078 (+ 1430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:20.958764 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:20.958881 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:20.959033 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0807 18:37:20.959177 (+ 144us) server_negotiation.cc:299] Negotiation successful
0807 18:37:20.959276 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":830,"thread_start_us":81,"threads_started":1}
I20260807 18:37:20.959412 32752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:20.953280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.65:44953 (local address 127.0.0.1:34364)
0807 18:37:20.953923 (+ 643us) negotiation.cc:107] Waiting for socket to connect
0807 18:37:20.953974 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0807 18:37:20.954095 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:37:20.954923 (+ 828us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:37:20.954929 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:37:20.954954 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:37:20.955103 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:37:20.955113 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:37:20.957029 (+ 1916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:37:20.957032 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:37:20.957948 (+ 916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:37:20.957958 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:37:20.958886 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:37:20.958889 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:37:20.958967 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:20.958998 (+ 31us) client_negotiation.cc:769] Sending connection context
0807 18:37:20.959137 (+ 139us) client_negotiation.cc:240] Negotiation successful
0807 18:37:20.959260 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":463,"mutex_wait_us":6,"thread_start_us":95,"threads_started":1}
I20260807 18:37:20.959960 32756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:20.954398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.67:36863 (local address 127.0.0.1:55518)
0807 18:37:20.954693 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0807 18:37:20.954706 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 18:37:20.954796 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:37:20.955584 (+ 788us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:37:20.955587 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:37:20.955604 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:37:20.955690 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:37:20.955696 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:37:20.956724 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:37:20.956727 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:37:20.958457 (+ 1730us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:37:20.958469 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:37:20.959538 (+ 1069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:37:20.959542 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:37:20.959661 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:20.959687 (+ 26us) client_negotiation.cc:769] Sending connection context
0807 18:37:20.959741 (+ 54us) client_negotiation.cc:240] Negotiation successful
0807 18:37:20.959823 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":238,"mutex_wait_us":166,"thread_start_us":97,"threads_started":1}
I20260807 18:37:20.960574 32757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:20.954684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55518 (local address 127.11.14.67:36863)
0807 18:37:20.955110 (+ 426us) server_negotiation.cc:206] Beginning negotiation
0807 18:37:20.955123 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 18:37:20.955143 (+ 20us) server_negotiation.cc:407] Connection header received
0807 18:37:20.955215 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:37:20.955221 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:37:20.955434 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:37:20.955622 (+ 188us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:37:20.955807 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:20.956570 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:20.958592 (+ 2022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:20.959400 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:20.960136 (+ 736us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:20.960255 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0807 18:37:20.960309 (+ 54us) server_negotiation.cc:299] Negotiation successful
0807 18:37:20.960402 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":326,"thread_start_us":239,"threads_started":1}
I20260807 18:37:20.960711 32753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:20.953348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.66:45307 (local address 127.0.0.1:53952)
0807 18:37:20.954347 (+ 999us) negotiation.cc:107] Waiting for socket to connect
0807 18:37:20.954361 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 18:37:20.954450 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:37:20.955126 (+ 676us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:37:20.955132 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:37:20.955151 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:37:20.955254 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:37:20.955260 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:37:20.956812 (+ 1552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:37:20.956816 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:37:20.958457 (+ 1641us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:37:20.958469 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:37:20.960319 (+ 1850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:37:20.960323 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:37:20.960408 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:20.960429 (+ 21us) client_negotiation.cc:769] Sending connection context
0807 18:37:20.960508 (+ 79us) client_negotiation.cc:240] Negotiation successful
0807 18:37:20.960573 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":942,"mutex_wait_us":155,"thread_start_us":107,"threads_started":1}
I20260807 18:37:20.960825 32755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:20.953754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53952 (local address 127.11.14.66:45307)
0807 18:37:20.954051 (+ 297us) server_negotiation.cc:206] Beginning negotiation
0807 18:37:20.954063 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 18:37:20.954546 (+ 483us) server_negotiation.cc:407] Connection header received
0807 18:37:20.954623 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:37:20.954627 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:37:20.954782 (+ 155us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:37:20.954967 (+ 185us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:37:20.955396 (+ 429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:20.956649 (+ 1253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:20.959107 (+ 2458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:20.959977 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:20.960098 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:20.960242 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0807 18:37:20.960543 (+ 301us) server_negotiation.cc:299] Negotiation successful
0807 18:37:20.960641 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":199,"thread_start_us":114,"threads_started":1}
I20260807 18:37:20.963189 32371 tablet_service.cc:1467] Processing CreateTablet for tablet 139ed10f01d449fcaebff0ab51e1a65d (DEFAULT_TABLE table=test-table [id=d940ad3067964bed97bea48eb2767ef1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:37:20.963526 32518 tablet_service.cc:1467] Processing CreateTablet for tablet 139ed10f01d449fcaebff0ab51e1a65d (DEFAULT_TABLE table=test-table [id=d940ad3067964bed97bea48eb2767ef1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:37:20.964452 32371 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 139ed10f01d449fcaebff0ab51e1a65d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:37:20.964612 32518 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 139ed10f01d449fcaebff0ab51e1a65d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:37:20.964547 32665 tablet_service.cc:1467] Processing CreateTablet for tablet 139ed10f01d449fcaebff0ab51e1a65d (DEFAULT_TABLE table=test-table [id=d940ad3067964bed97bea48eb2767ef1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:37:20.965520 32665 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 139ed10f01d449fcaebff0ab51e1a65d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:37:20.970643 32758 tablet_bootstrap.cc:492] T 139ed10f01d449fcaebff0ab51e1a65d P 7baa370b5aad4902a6e7cb2ac21d8ab7: Bootstrap starting.
I20260807 18:37:20.972090 32759 tablet_bootstrap.cc:492] T 139ed10f01d449fcaebff0ab51e1a65d P 5c53f91c64bc4245a7320bad5c78ed76: Bootstrap starting.
I20260807 18:37:20.972834 32760 tablet_bootstrap.cc:492] T 139ed10f01d449fcaebff0ab51e1a65d P 5b957e65b83347f3a1f43b8005355979: Bootstrap starting.
I20260807 18:37:20.973758 32758 tablet_bootstrap.cc:654] T 139ed10f01d449fcaebff0ab51e1a65d P 7baa370b5aad4902a6e7cb2ac21d8ab7: Neither blocks nor log segments found. Creating new log.
I20260807 18:37:20.974670 32759 tablet_bootstrap.cc:654] T 139ed10f01d449fcaebff0ab51e1a65d P 5c53f91c64bc4245a7320bad5c78ed76: Neither blocks nor log segments found. Creating new log.
I20260807 18:37:20.974722 32760 tablet_bootstrap.cc:654] T 139ed10f01d449fcaebff0ab51e1a65d P 5b957e65b83347f3a1f43b8005355979: Neither blocks nor log segments found. Creating new log.
I20260807 18:37:20.974948 32758 log.cc:824] T 139ed10f01d449fcaebff0ab51e1a65d P 7baa370b5aad4902a6e7cb2ac21d8ab7: Log is configured to *not* fsync() on all Append() calls
I20260807 18:37:20.975521 32759 log.cc:824] T 139ed10f01d449fcaebff0ab51e1a65d P 5c53f91c64bc4245a7320bad5c78ed76: Log is configured to *not* fsync() on all Append() calls
I20260807 18:37:20.975980 32760 log.cc:824] T 139ed10f01d449fcaebff0ab51e1a65d P 5b957e65b83347f3a1f43b8005355979: Log is configured to *not* fsync() on all Append() calls
I20260807 18:37:20.977324 32758 tablet_bootstrap.cc:492] T 139ed10f01d449fcaebff0ab51e1a65d P 7baa370b5aad4902a6e7cb2ac21d8ab7: No bootstrap required, opened a new log
I20260807 18:37:20.977530 32758 ts_tablet_manager.cc:1397] T 139ed10f01d449fcaebff0ab51e1a65d P 7baa370b5aad4902a6e7cb2ac21d8ab7: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260807 18:37:20.977593 32759 tablet_bootstrap.cc:492] T 139ed10f01d449fcaebff0ab51e1a65d P 5c53f91c64bc4245a7320bad5c78ed76: No bootstrap required, opened a new log
I20260807 18:37:20.977757 32759 ts_tablet_manager.cc:1397] T 139ed10f01d449fcaebff0ab51e1a65d P 5c53f91c64bc4245a7320bad5c78ed76: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260807 18:37:20.980969 32760 tablet_bootstrap.cc:492] T 139ed10f01d449fcaebff0ab51e1a65d P 5b957e65b83347f3a1f43b8005355979: No bootstrap required, opened a new log
I20260807 18:37:20.981150 32760 ts_tablet_manager.cc:1397] T 139ed10f01d449fcaebff0ab51e1a65d P 5b957e65b83347f3a1f43b8005355979: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.000s
I20260807 18:37:20.981799 32759 raft_consensus.cc:348] T 139ed10f01d449fcaebff0ab51e1a65d P 5c53f91c64bc4245a7320bad5c78ed76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7baa370b5aad4902a6e7cb2ac21d8ab7" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 44953 } } peers { permanent_uuid: "5c53f91c64bc4245a7320bad5c78ed76" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 45307 } } peers { permanent_uuid: "5b957e65b83347f3a1f43b8005355979" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 36863 } }
I20260807 18:37:20.982108 32759 raft_consensus.cc:374] T 139ed10f01d449fcaebff0ab51e1a65d P 5c53f91c64bc4245a7320bad5c78ed76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:37:20.981966 32758 raft_consensus.cc:348] T 139ed10f01d449fcaebff0ab51e1a65d P 7baa370b5aad4902a6e7cb2ac21d8ab7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7baa370b5aad4902a6e7cb2ac21d8ab7" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 44953 } } peers { permanent_uuid: "5c53f91c64bc4245a7320bad5c78ed76" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 45307 } } peers { permanent_uuid: "5b957e65b83347f3a1f43b8005355979" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 36863 } }
I20260807 18:37:20.982167 32759 raft_consensus.cc:729] T 139ed10f01d449fcaebff0ab51e1a65d P 5c53f91c64bc4245a7320bad5c78ed76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c53f91c64bc4245a7320bad5c78ed76, State: Initialized, Role: FOLLOWER
I20260807 18:37:20.982189 32758 raft_consensus.cc:374] T 139ed10f01d449fcaebff0ab51e1a65d P 7baa370b5aad4902a6e7cb2ac21d8ab7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:37:20.982246 32758 raft_consensus.cc:729] T 139ed10f01d449fcaebff0ab51e1a65d P 7baa370b5aad4902a6e7cb2ac21d8ab7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7baa370b5aad4902a6e7cb2ac21d8ab7, State: Initialized, Role: FOLLOWER
I20260807 18:37:20.982671 32758 consensus_queue.cc:260] T 139ed10f01d449fcaebff0ab51e1a65d P 7baa370b5aad4902a6e7cb2ac21d8ab7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7baa370b5aad4902a6e7cb2ac21d8ab7" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 44953 } } peers { permanent_uuid: "5c53f91c64bc4245a7320bad5c78ed76" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 45307 } } peers { permanent_uuid: "5b957e65b83347f3a1f43b8005355979" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 36863 } }
I20260807 18:37:20.982671 32759 consensus_queue.cc:260] T 139ed10f01d449fcaebff0ab51e1a65d P 5c53f91c64bc4245a7320bad5c78ed76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7baa370b5aad4902a6e7cb2ac21d8ab7" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 44953 } } peers { permanent_uuid: "5c53f91c64bc4245a7320bad5c78ed76" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 45307 } } peers { permanent_uuid: "5b957e65b83347f3a1f43b8005355979" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 36863 } }
I20260807 18:37:20.983599 32589 heartbeater.cc:503] Master 127.11.14.126:35917 was elected leader, sending a full tablet report...
I20260807 18:37:20.983737 32758 ts_tablet_manager.cc:1428] T 139ed10f01d449fcaebff0ab51e1a65d P 7baa370b5aad4902a6e7cb2ac21d8ab7: Time spent starting tablet: real 0.006s user 0.008s sys 0.000s
I20260807 18:37:20.983737 32759 ts_tablet_manager.cc:1428] T 139ed10f01d449fcaebff0ab51e1a65d P 5c53f91c64bc4245a7320bad5c78ed76: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260807 18:37:20.984066 32447 heartbeater.cc:503] Master 127.11.14.126:35917 was elected leader, sending a full tablet report...
I20260807 18:37:20.984915 32760 raft_consensus.cc:348] T 139ed10f01d449fcaebff0ab51e1a65d P 5b957e65b83347f3a1f43b8005355979 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7baa370b5aad4902a6e7cb2ac21d8ab7" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 44953 } } peers { permanent_uuid: "5c53f91c64bc4245a7320bad5c78ed76" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 45307 } } peers { permanent_uuid: "5b957e65b83347f3a1f43b8005355979" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 36863 } }
I20260807 18:37:20.985195 32760 raft_consensus.cc:374] T 139ed10f01d449fcaebff0ab51e1a65d P 5b957e65b83347f3a1f43b8005355979 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:37:20.985260 32760 raft_consensus.cc:729] T 139ed10f01d449fcaebff0ab51e1a65d P 5b957e65b83347f3a1f43b8005355979 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b957e65b83347f3a1f43b8005355979, State: Initialized, Role: FOLLOWER
I20260807 18:37:20.985641 32760 consensus_queue.cc:260] T 139ed10f01d449fcaebff0ab51e1a65d P 5b957e65b83347f3a1f43b8005355979 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7baa370b5aad4902a6e7cb2ac21d8ab7" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 44953 } } peers { permanent_uuid: "5c53f91c64bc4245a7320bad5c78ed76" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 45307 } } peers { permanent_uuid: "5b957e65b83347f3a1f43b8005355979" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 36863 } }
I20260807 18:37:20.986197 32740 heartbeater.cc:503] Master 127.11.14.126:35917 was elected leader, sending a full tablet report...
I20260807 18:37:20.986593 32760 ts_tablet_manager.cc:1428] T 139ed10f01d449fcaebff0ab51e1a65d P 5b957e65b83347f3a1f43b8005355979: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260807 18:37:21.018996 32766 raft_consensus.cc:482] T 139ed10f01d449fcaebff0ab51e1a65d P 5b957e65b83347f3a1f43b8005355979 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:37:21.019237 32766 raft_consensus.cc:504] T 139ed10f01d449fcaebff0ab51e1a65d P 5b957e65b83347f3a1f43b8005355979 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7baa370b5aad4902a6e7cb2ac21d8ab7" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 44953 } } peers { permanent_uuid: "5c53f91c64bc4245a7320bad5c78ed76" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 45307 } } peers { permanent_uuid: "5b957e65b83347f3a1f43b8005355979" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 36863 } }
I20260807 18:37:21.020579 32766 leader_election.cc:288] T 139ed10f01d449fcaebff0ab51e1a65d P 5b957e65b83347f3a1f43b8005355979 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7baa370b5aad4902a6e7cb2ac21d8ab7 (127.11.14.65:44953), 5c53f91c64bc4245a7320bad5c78ed76 (127.11.14.66:45307)
I20260807 18:37:21.024331 32742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:21.020762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.65:44953 (local address 127.11.14.67:58975)
0807 18:37:21.020979 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0807 18:37:21.020997 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 18:37:21.021093 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:37:21.021400 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:37:21.021403 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:37:21.021421 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:37:21.021547 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:37:21.021553 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:37:21.022283 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:37:21.022288 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:37:21.023164 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:37:21.023174 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:37:21.023880 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:37:21.023883 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:37:21.023987 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:21.024016 (+ 29us) client_negotiation.cc:769] Sending connection context
0807 18:37:21.024111 (+ 95us) client_negotiation.cc:240] Negotiation successful
0807 18:37:21.024190 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"mutex_wait_us":29}
I20260807 18:37:21.024331 32754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:21.020920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.67:58975 (local address 127.11.14.65:44953)
0807 18:37:21.021068 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0807 18:37:21.021072 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:37:21.021088 (+ 16us) server_negotiation.cc:407] Connection header received
0807 18:37:21.021230 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:37:21.021233 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:37:21.021295 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:37:21.021384 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:37:21.021659 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:21.022163 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:21.023292 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:21.023751 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:21.023887 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:21.023941 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0807 18:37:21.024128 (+ 187us) server_negotiation.cc:299] Negotiation successful
0807 18:37:21.024212 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260807 18:37:21.024696 32755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:21.021019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.67:45721 (local address 127.11.14.66:45307)
0807 18:37:21.021150 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0807 18:37:21.021155 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:37:21.021371 (+ 216us) server_negotiation.cc:407] Connection header received
0807 18:37:21.021526 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:37:21.021529 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:37:21.021575 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:37:21.021664 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:37:21.021940 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:21.022418 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:21.023503 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:21.024079 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:21.024202 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:21.024252 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0807 18:37:21.024470 (+ 218us) server_negotiation.cc:299] Negotiation successful
0807 18:37:21.024560 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260807 18:37:21.025070 32767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:21.020841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.66:45307 (local address 127.11.14.67:45721)
0807 18:37:21.021261 (+ 420us) negotiation.cc:107] Waiting for socket to connect
0807 18:37:21.021273 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 18:37:21.021406 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:37:21.021671 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:37:21.021674 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:37:21.021690 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:37:21.021825 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:37:21.021831 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:37:21.022527 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:37:21.022532 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:37:21.023369 (+ 837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:37:21.023388 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:37:21.024202 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:37:21.024205 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:37:21.024317 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:21.024338 (+ 21us) client_negotiation.cc:769] Sending connection context
0807 18:37:21.024881 (+ 543us) client_negotiation.cc:240] Negotiation successful
0807 18:37:21.024962 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":370,"mutex_wait_us":40,"thread_start_us":133,"threads_started":1}
I20260807 18:37:21.025223 32395 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "139ed10f01d449fcaebff0ab51e1a65d" candidate_uuid: "5b957e65b83347f3a1f43b8005355979" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7baa370b5aad4902a6e7cb2ac21d8ab7" is_pre_election: true
I20260807 18:37:21.025559 32395 raft_consensus.cc:2454] T 139ed10f01d449fcaebff0ab51e1a65d P 7baa370b5aad4902a6e7cb2ac21d8ab7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b957e65b83347f3a1f43b8005355979 in term 0.
I20260807 18:37:21.025713 32540 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "139ed10f01d449fcaebff0ab51e1a65d" candidate_uuid: "5b957e65b83347f3a1f43b8005355979" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c53f91c64bc4245a7320bad5c78ed76" is_pre_election: true
I20260807 18:37:21.026029 32540 raft_consensus.cc:2454] T 139ed10f01d449fcaebff0ab51e1a65d P 5c53f91c64bc4245a7320bad5c78ed76 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b957e65b83347f3a1f43b8005355979 in term 0.
I20260807 18:37:21.026119 32614 leader_election.cc:302] T 139ed10f01d449fcaebff0ab51e1a65d P 5b957e65b83347f3a1f43b8005355979 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b957e65b83347f3a1f43b8005355979, 7baa370b5aad4902a6e7cb2ac21d8ab7; no voters:
I20260807 18:37:21.026372 32766 raft_consensus.cc:2781] T 139ed10f01d449fcaebff0ab51e1a65d P 5b957e65b83347f3a1f43b8005355979 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:37:21.026458 32766 raft_consensus.cc:482] T 139ed10f01d449fcaebff0ab51e1a65d P 5b957e65b83347f3a1f43b8005355979 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:37:21.026510 32766 raft_consensus.cc:3037] T 139ed10f01d449fcaebff0ab51e1a65d P 5b957e65b83347f3a1f43b8005355979 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:37:21.027601 32766 raft_consensus.cc:504] T 139ed10f01d449fcaebff0ab51e1a65d P 5b957e65b83347f3a1f43b8005355979 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7baa370b5aad4902a6e7cb2ac21d8ab7" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 44953 } } peers { permanent_uuid: "5c53f91c64bc4245a7320bad5c78ed76" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 45307 } } peers { permanent_uuid: "5b957e65b83347f3a1f43b8005355979" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 36863 } }
I20260807 18:37:21.028008 32766 leader_election.cc:288] T 139ed10f01d449fcaebff0ab51e1a65d P 5b957e65b83347f3a1f43b8005355979 [CANDIDATE]: Term 1 election: Requested vote from peers 7baa370b5aad4902a6e7cb2ac21d8ab7 (127.11.14.65:44953), 5c53f91c64bc4245a7320bad5c78ed76 (127.11.14.66:45307)
I20260807 18:37:21.028330 32395 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "139ed10f01d449fcaebff0ab51e1a65d" candidate_uuid: "5b957e65b83347f3a1f43b8005355979" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7baa370b5aad4902a6e7cb2ac21d8ab7"
I20260807 18:37:21.028501 32395 raft_consensus.cc:3037] T 139ed10f01d449fcaebff0ab51e1a65d P 7baa370b5aad4902a6e7cb2ac21d8ab7 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:37:21.028553 32540 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "139ed10f01d449fcaebff0ab51e1a65d" candidate_uuid: "5b957e65b83347f3a1f43b8005355979" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c53f91c64bc4245a7320bad5c78ed76"
I20260807 18:37:21.028662 32540 raft_consensus.cc:3037] T 139ed10f01d449fcaebff0ab51e1a65d P 5c53f91c64bc4245a7320bad5c78ed76 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:37:21.029621 32395 raft_consensus.cc:2454] T 139ed10f01d449fcaebff0ab51e1a65d P 7baa370b5aad4902a6e7cb2ac21d8ab7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b957e65b83347f3a1f43b8005355979 in term 1.
I20260807 18:37:21.029856 32540 raft_consensus.cc:2454] T 139ed10f01d449fcaebff0ab51e1a65d P 5c53f91c64bc4245a7320bad5c78ed76 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b957e65b83347f3a1f43b8005355979 in term 1.
I20260807 18:37:21.029932 32614 leader_election.cc:302] T 139ed10f01d449fcaebff0ab51e1a65d P 5b957e65b83347f3a1f43b8005355979 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b957e65b83347f3a1f43b8005355979, 7baa370b5aad4902a6e7cb2ac21d8ab7; no voters:
I20260807 18:37:21.030117 32766 raft_consensus.cc:2781] T 139ed10f01d449fcaebff0ab51e1a65d P 5b957e65b83347f3a1f43b8005355979 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:37:21.030388 32766 raft_consensus.cc:686] T 139ed10f01d449fcaebff0ab51e1a65d P 5b957e65b83347f3a1f43b8005355979 [term 1 LEADER]: Becoming Leader. State: Replica: 5b957e65b83347f3a1f43b8005355979, State: Running, Role: LEADER
I20260807 18:37:21.030735 32766 consensus_queue.cc:237] T 139ed10f01d449fcaebff0ab51e1a65d P 5b957e65b83347f3a1f43b8005355979 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7baa370b5aad4902a6e7cb2ac21d8ab7" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 44953 } } peers { permanent_uuid: "5c53f91c64bc4245a7320bad5c78ed76" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 45307 } } peers { permanent_uuid: "5b957e65b83347f3a1f43b8005355979" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 36863 } }
I20260807 18:37:21.033986 32228 catalog_manager.cc:5495] T 139ed10f01d449fcaebff0ab51e1a65d P 5b957e65b83347f3a1f43b8005355979 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5b957e65b83347f3a1f43b8005355979 (127.11.14.67). New cstate: current_term: 1 leader_uuid: "5b957e65b83347f3a1f43b8005355979" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7baa370b5aad4902a6e7cb2ac21d8ab7" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 44953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5c53f91c64bc4245a7320bad5c78ed76" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 45307 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5b957e65b83347f3a1f43b8005355979" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 36863 } health_report { overall_health: HEALTHY } } }
I20260807 18:37:21.057826 32757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:21.054156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55522 (local address 127.11.14.67:36863)
0807 18:37:21.054290 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0807 18:37:21.054294 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:37:21.054309 (+ 15us) server_negotiation.cc:407] Connection header received
0807 18:37:21.054485 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:37:21.054487 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:37:21.054542 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:37:21.054653 (+ 111us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:37:21.054921 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:21.055428 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:21.056327 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:21.056545 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:21.056655 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:21.056974 (+ 319us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:37:21.057399 (+ 425us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:37:21.057541 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0807 18:37:21.057585 (+ 44us) server_negotiation.cc:299] Negotiation successful
0807 18:37:21.057643 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260807 18:37:21.064105 32540 raft_consensus.cc:1261] T 139ed10f01d449fcaebff0ab51e1a65d P 5c53f91c64bc4245a7320bad5c78ed76 [term 1 FOLLOWER]: Refusing update from remote peer 5b957e65b83347f3a1f43b8005355979: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:37:21.064183 32395 raft_consensus.cc:1261] T 139ed10f01d449fcaebff0ab51e1a65d P 7baa370b5aad4902a6e7cb2ac21d8ab7 [term 1 FOLLOWER]: Refusing update from remote peer 5b957e65b83347f3a1f43b8005355979: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:37:21.064857 300 consensus_queue.cc:1035] T 139ed10f01d449fcaebff0ab51e1a65d P 5b957e65b83347f3a1f43b8005355979 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c53f91c64bc4245a7320bad5c78ed76" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 45307 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:37:21.065053 32766 consensus_queue.cc:1035] T 139ed10f01d449fcaebff0ab51e1a65d P 5b957e65b83347f3a1f43b8005355979 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7baa370b5aad4902a6e7cb2ac21d8ab7" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 44953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:37:21.074479 304 mvcc.cc:205] Tried to move back new op lower bound from 7315979636990066688 to 7315979636864815104. Current Snapshot: MvccSnapshot[applied={T|T < 7315979636990066688}]
I20260807 18:37:21.080701 32226 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56610:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:37:21.080885 32226 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56610:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:37:21.083310 32226 catalog_manager.cc:5782] T 00000000000000000000000000000000 P efd7333152f742efb3ed32d10d6539b2: Sending DeleteTablet for 3 replicas of tablet 139ed10f01d449fcaebff0ab51e1a65d
I20260807 18:37:21.084108 32371 tablet_service.cc:1514] Processing DeleteTablet for tablet 139ed10f01d449fcaebff0ab51e1a65d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:37:21 UTC) from {username='kudu', principal='kudu/127.11.14.126@KRBTEST.COM'} at 127.0.0.1:34364
I20260807 18:37:21.084165 32518 tablet_service.cc:1514] Processing DeleteTablet for tablet 139ed10f01d449fcaebff0ab51e1a65d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:37:21 UTC) from {username='kudu', principal='kudu/127.11.14.126@KRBTEST.COM'} at 127.0.0.1:53952
I20260807 18:37:21.084321 32665 tablet_service.cc:1514] Processing DeleteTablet for tablet 139ed10f01d449fcaebff0ab51e1a65d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:37:21 UTC) from {username='kudu', principal='kudu/127.11.14.126@KRBTEST.COM'} at 127.0.0.1:55518
I20260807 18:37:21.084765 313 tablet_replica.cc:330] T 139ed10f01d449fcaebff0ab51e1a65d P 7baa370b5aad4902a6e7cb2ac21d8ab7: stopping tablet replica
I20260807 18:37:21.084764 315 tablet_replica.cc:330] T 139ed10f01d449fcaebff0ab51e1a65d P 5b957e65b83347f3a1f43b8005355979: stopping tablet replica
I20260807 18:37:21.084906 11321 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkz56XN/build/debug/bin/kudu with pid 32301
I20260807 18:37:21.085078 315 raft_consensus.cc:2229] T 139ed10f01d449fcaebff0ab51e1a65d P 5b957e65b83347f3a1f43b8005355979 [term 1 LEADER]: Raft consensus shutting down.
I20260807 18:37:21.085078 313 raft_consensus.cc:2229] T 139ed10f01d449fcaebff0ab51e1a65d P 7baa370b5aad4902a6e7cb2ac21d8ab7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:37:21.085610 315 raft_consensus.cc:2258] T 139ed10f01d449fcaebff0ab51e1a65d P 5b957e65b83347f3a1f43b8005355979 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:37:21.088213 314 tablet_replica.cc:330] T 139ed10f01d449fcaebff0ab51e1a65d P 5c53f91c64bc4245a7320bad5c78ed76: stopping tablet replica
I20260807 18:37:21.088264 315 ts_tablet_manager.cc:1905] T 139ed10f01d449fcaebff0ab51e1a65d P 5b957e65b83347f3a1f43b8005355979: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:37:21.088536 314 raft_consensus.cc:2229] T 139ed10f01d449fcaebff0ab51e1a65d P 5c53f91c64bc4245a7320bad5c78ed76 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:37:21.088871 314 raft_consensus.cc:2258] T 139ed10f01d449fcaebff0ab51e1a65d P 5c53f91c64bc4245a7320bad5c78ed76 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:37:21.090394 314 ts_tablet_manager.cc:1905] T 139ed10f01d449fcaebff0ab51e1a65d P 5c53f91c64bc4245a7320bad5c78ed76: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:37:21.091279 315 ts_tablet_manager.cc:1918] T 139ed10f01d449fcaebff0ab51e1a65d P 5b957e65b83347f3a1f43b8005355979: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:37:21.091385 315 log.cc:1196] T 139ed10f01d449fcaebff0ab51e1a65d P 5b957e65b83347f3a1f43b8005355979: Deleting WAL directory at /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127579900523-11321-0/minicluster-data/ts-2/wal/wals/139ed10f01d449fcaebff0ab51e1a65d
I20260807 18:37:21.091800 315 ts_tablet_manager.cc:1939] T 139ed10f01d449fcaebff0ab51e1a65d P 5b957e65b83347f3a1f43b8005355979: Deleting consensus metadata
I20260807 18:37:21.093125 32212 catalog_manager.cc:4841] TS 5b957e65b83347f3a1f43b8005355979 (127.11.14.67:36863): tablet 139ed10f01d449fcaebff0ab51e1a65d (table test-table [id=d940ad3067964bed97bea48eb2767ef1]) successfully deleted
I20260807 18:37:21.093292 314 ts_tablet_manager.cc:1918] T 139ed10f01d449fcaebff0ab51e1a65d P 5c53f91c64bc4245a7320bad5c78ed76: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:37:21.093397 314 log.cc:1196] T 139ed10f01d449fcaebff0ab51e1a65d P 5c53f91c64bc4245a7320bad5c78ed76: Deleting WAL directory at /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786127579900523-11321-0/minicluster-data/ts-1/wal/wals/139ed10f01d449fcaebff0ab51e1a65d
I20260807 18:37:21.093701 314 ts_tablet_manager.cc:1939] T 139ed10f01d449fcaebff0ab51e1a65d P 5c53f91c64bc4245a7320bad5c78ed76: Deleting consensus metadata
W20260807 18:37:21.094636 32211 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.14.65:44953 (error 108)
I20260807 18:37:21.094923 32213 catalog_manager.cc:4841] TS 5c53f91c64bc4245a7320bad5c78ed76 (127.11.14.66:45307): tablet 139ed10f01d449fcaebff0ab51e1a65d (table test-table [id=d940ad3067964bed97bea48eb2767ef1]) successfully deleted
I20260807 18:37:21.095743 32752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:21.095405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.65:44953 (local address 127.0.0.1:34380)
0807 18:37:21.095525 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0807 18:37:21.095636 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.65:44953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260807 18:37:21.095868 11321 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkz56XN/build/debug/bin/kudu with pid 32453
W20260807 18:37:21.101080 32211 catalog_manager.cc:4584] TS 7baa370b5aad4902a6e7cb2ac21d8ab7 (127.11.14.65:44953): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 139ed10f01d449fcaebff0ab51e1a65d: Network error: Client connection negotiation failed: client connection to 127.11.14.65:44953: connect: Connection refused (error 111)
I20260807 18:37:21.107016 11321 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkz56XN/build/debug/bin/kudu with pid 32595
I20260807 18:37:21.115849 11321 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkz56XN/build/debug/bin/kudu with pid 32193
2026-08-07T18:37:21Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2392 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-07T18:37:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:37:21Z Disabled control of system clock
I20260807 18:37:21.150408 11321 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkz56XN/build/debug/bin/kudu
/tmp/dist-test-taskkz56XN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786127579900523-11321-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786127579900523-11321-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786127579900523-11321-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786127579900523-11321-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.14.126:33763
--webserver_interface=127.11.14.126
--webserver_port=0
--builtin_ntp_servers=127.11.14.84:41225
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.14.126:33763
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 18:37:21.268849 320 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:37:21.269132 320 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:37:21.269170 320 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:37:21.273252 320 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:37:21.273322 320 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:37:21.273373 320 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:37:21.273412 320 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:37:21.273449 320 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:37:21.278431 320 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.14.84:41225
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786127579900523-11321-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786127579900523-11321-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.14.126:33763
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.14.126:33763
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786127579900523-11321-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.14.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.320
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786127579900523-11321-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:37:21.279654 320 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:37:21.281111 320 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:37:21.289721 329 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:37:21.289731 325 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:37:21.289737 327 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:37:21.290094 320 server_base.cc:1034] running on GCE node
I20260807 18:37:21.290594 320 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:37:21.291709 320 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:37:21.292869 320 hybrid_clock.cc:648] HybridClock initialized: now 1786127841292851 us; error 32 us; skew 500 ppm
I20260807 18:37:21.294847 320 webserver.cc:460] Webserver started at http://127.11.14.126:38797/ using document root <none> and password file <none>
I20260807 18:37:21.295424 320 fs_manager.cc:359] Metadata directory not provided
I20260807 18:37:21.295486 320 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:37:21.295727 320 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:37:21.297782 320 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786127579900523-11321-0/minicluster-data/master-0/data/instance:
uuid: "224a5b31ef3442d7a9cb3f8b42b80736"
format_stamp: "Formatted at 2026-08-07 18:37:21 on dist-test-slave-6k22"
server_key: "a775a0e2824618cd17b54885d0269b72"
server_key_iv: "9f099ee52ba933c47ec78f2c7f5adff0"
server_key_version: "encryptionkey@0"
I20260807 18:37:21.298295 320 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786127579900523-11321-0/minicluster-data/master-0/wal/instance:
uuid: "224a5b31ef3442d7a9cb3f8b42b80736"
format_stamp: "Formatted at 2026-08-07 18:37:21 on dist-test-slave-6k22"
server_key: "a775a0e2824618cd17b54885d0269b72"
server_key_iv: "9f099ee52ba933c47ec78f2c7f5adff0"
server_key_version: "encryptionkey@0"
I20260807 18:37:21.301851 320 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260807 18:37:21.304144 335 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:37:21.305358 320 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 18:37:21.305495 320 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786127579900523-11321-0/minicluster-data/master-0/data,/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786127579900523-11321-0/minicluster-data/master-0/wal
uuid: "224a5b31ef3442d7a9cb3f8b42b80736"
format_stamp: "Formatted at 2026-08-07 18:37:21 on dist-test-slave-6k22"
server_key: "a775a0e2824618cd17b54885d0269b72"
server_key_iv: "9f099ee52ba933c47ec78f2c7f5adff0"
server_key_version: "encryptionkey@0"
I20260807 18:37:21.305600 320 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786127579900523-11321-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786127579900523-11321-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786127579900523-11321-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:37:21.325531 320 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:37:21.326278 320 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:37:21.326469 320 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:37:21.333472 320 rpc_server.cc:307] RPC server started. Bound to: 127.11.14.126:33763
I20260807 18:37:21.333487 394 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.14.126:33763 every 8 connection(s)
I20260807 18:37:21.334407 320 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786127579900523-11321-0/minicluster-data/master-0/data/info.pb
I20260807 18:37:21.336618 11321 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkz56XN/build/debug/bin/kudu as pid 320
I20260807 18:37:21.336760 11321 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786127579900523-11321-0/minicluster-data/master-0/wal/instance
I20260807 18:37:21.337049 11321 external_mini_cluster.cc:1428] Setting key 8d5f8ac8a86c32e73d9f62affa0cb158
I20260807 18:37:21.337824 395 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:37:21.345059 395 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 224a5b31ef3442d7a9cb3f8b42b80736: Bootstrap starting.
I20260807 18:37:21.347879 399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:21.338554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38514 (local address 127.11.14.126:33763)
0807 18:37:21.339239 (+ 685us) server_negotiation.cc:206] Beginning negotiation
0807 18:37:21.339261 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0807 18:37:21.339308 (+ 47us) server_negotiation.cc:407] Connection header received
0807 18:37:21.340176 (+ 868us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:37:21.340200 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:37:21.340701 (+ 501us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:37:21.341085 (+ 384us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:37:21.341536 (+ 451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:21.342972 (+ 1436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:21.344019 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:21.344689 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:21.344863 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:21.345767 (+ 904us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:37:21.345781 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:37:21.345796 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:37:21.346286 (+ 490us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:37:21.346361 (+ 75us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:37:21.346391 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:37:21.346498 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0807 18:37:21.346650 (+ 152us) server_negotiation.cc:299] Negotiation successful
0807 18:37:21.346925 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":294,"thread_start_us":114,"threads_started":1}
I20260807 18:37:21.347895 395 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 224a5b31ef3442d7a9cb3f8b42b80736: Neither blocks nor log segments found. Creating new log.
I20260807 18:37:21.348806 395 log.cc:824] T 00000000000000000000000000000000 P 224a5b31ef3442d7a9cb3f8b42b80736: Log is configured to *not* fsync() on all Append() calls
I20260807 18:37:21.350821 395 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 224a5b31ef3442d7a9cb3f8b42b80736: No bootstrap required, opened a new log
I20260807 18:37:21.354259 395 raft_consensus.cc:348] T 00000000000000000000000000000000 P 224a5b31ef3442d7a9cb3f8b42b80736 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "224a5b31ef3442d7a9cb3f8b42b80736" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 33763 } }
I20260807 18:37:21.354482 395 raft_consensus.cc:374] T 00000000000000000000000000000000 P 224a5b31ef3442d7a9cb3f8b42b80736 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:37:21.354516 395 raft_consensus.cc:729] T 00000000000000000000000000000000 P 224a5b31ef3442d7a9cb3f8b42b80736 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 224a5b31ef3442d7a9cb3f8b42b80736, State: Initialized, Role: FOLLOWER
I20260807 18:37:21.354918 395 consensus_queue.cc:260] T 00000000000000000000000000000000 P 224a5b31ef3442d7a9cb3f8b42b80736 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "224a5b31ef3442d7a9cb3f8b42b80736" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 33763 } }
I20260807 18:37:21.355041 395 raft_consensus.cc:388] T 00000000000000000000000000000000 P 224a5b31ef3442d7a9cb3f8b42b80736 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:37:21.355085 395 raft_consensus.cc:482] T 00000000000000000000000000000000 P 224a5b31ef3442d7a9cb3f8b42b80736 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:37:21.355154 395 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 224a5b31ef3442d7a9cb3f8b42b80736 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:37:21.356029 395 raft_consensus.cc:504] T 00000000000000000000000000000000 P 224a5b31ef3442d7a9cb3f8b42b80736 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "224a5b31ef3442d7a9cb3f8b42b80736" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 33763 } }
I20260807 18:37:21.356325 395 leader_election.cc:302] T 00000000000000000000000000000000 P 224a5b31ef3442d7a9cb3f8b42b80736 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 224a5b31ef3442d7a9cb3f8b42b80736; no voters:
I20260807 18:37:21.356583 395 leader_election.cc:288] T 00000000000000000000000000000000 P 224a5b31ef3442d7a9cb3f8b42b80736 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:37:21.356724 401 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 224a5b31ef3442d7a9cb3f8b42b80736 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:37:21.356967 401 raft_consensus.cc:686] T 00000000000000000000000000000000 P 224a5b31ef3442d7a9cb3f8b42b80736 [term 1 LEADER]: Becoming Leader. State: Replica: 224a5b31ef3442d7a9cb3f8b42b80736, State: Running, Role: LEADER
I20260807 18:37:21.357265 401 consensus_queue.cc:237] T 00000000000000000000000000000000 P 224a5b31ef3442d7a9cb3f8b42b80736 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "224a5b31ef3442d7a9cb3f8b42b80736" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 33763 } }
I20260807 18:37:21.357851 395 sys_catalog.cc:564] T 00000000000000000000000000000000 P 224a5b31ef3442d7a9cb3f8b42b80736 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:37:21.358892 402 sys_catalog.cc:455] T 00000000000000000000000000000000 P 224a5b31ef3442d7a9cb3f8b42b80736 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "224a5b31ef3442d7a9cb3f8b42b80736" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "224a5b31ef3442d7a9cb3f8b42b80736" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 33763 } } }
I20260807 18:37:21.358870 403 sys_catalog.cc:455] T 00000000000000000000000000000000 P 224a5b31ef3442d7a9cb3f8b42b80736 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 224a5b31ef3442d7a9cb3f8b42b80736. Latest consensus state: current_term: 1 leader_uuid: "224a5b31ef3442d7a9cb3f8b42b80736" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "224a5b31ef3442d7a9cb3f8b42b80736" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 33763 } } }
I20260807 18:37:21.359026 402 sys_catalog.cc:458] T 00000000000000000000000000000000 P 224a5b31ef3442d7a9cb3f8b42b80736 [sys.catalog]: This master's current role is: LEADER
I20260807 18:37:21.359026 403 sys_catalog.cc:458] T 00000000000000000000000000000000 P 224a5b31ef3442d7a9cb3f8b42b80736 [sys.catalog]: This master's current role is: LEADER
I20260807 18:37:21.359463 410 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:37:21.362643 410 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:37:21.367887 410 catalog_manager.cc:1347] Generated new cluster ID: 0a7d28873a34450eb0cb4597dee12e39
I20260807 18:37:21.367959 410 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:37:21.375602 410 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:37:21.376509 410 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:37:21.381232 410 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 224a5b31ef3442d7a9cb3f8b42b80736: Generated new TSK 0
I20260807 18:37:21.381881 410 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:37:21.392045 11321 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkz56XN/build/debug/bin/kudu
/tmp/dist-test-taskkz56XN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786127579900523-11321-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786127579900523-11321-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786127579900523-11321-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786127579900523-11321-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.14.65:0
--local_ip_for_outbound_sockets=127.11.14.65
--webserver_interface=127.11.14.65
--webserver_port=0
--tserver_master_addrs=127.11.14.126:33763
--builtin_ntp_servers=127.11.14.84:41225
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 18:37:21.506832 422 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:37:21.507081 422 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:37:21.507136 422 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:37:21.511234 422 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:37:21.511287 422 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:37:21.511396 422 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.14.65
I20260807 18:37:21.516057 422 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.14.84:41225
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786127579900523-11321-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786127579900523-11321-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.14.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786127579900523-11321-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.14.65
--webserver_port=0
--tserver_master_addrs=127.11.14.126:33763
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.422
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.14.65
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786127579900523-11321-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:37:21.517117 422 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:37:21.518527 422 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:37:21.527999 431 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:37:21.528080 422 server_base.cc:1034] running on GCE node
W20260807 18:37:21.527995 428 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:37:21.528007 429 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:37:21.528663 422 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:37:21.529313 422 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:37:21.530490 422 hybrid_clock.cc:648] HybridClock initialized: now 1786127841530464 us; error 42 us; skew 500 ppm
I20260807 18:37:21.537839 422 webserver.cc:460] Webserver started at http://127.11.14.65:38051/ using document root <none> and password file <none>
I20260807 18:37:21.538412 422 fs_manager.cc:359] Metadata directory not provided
I20260807 18:37:21.538476 422 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:37:21.538648 422 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:37:21.540840 422 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786127579900523-11321-0/minicluster-data/ts-0/data/instance:
uuid: "919e53b0c66843af85ad640da2d6bcb2"
format_stamp: "Formatted at 2026-08-07 18:37:21 on dist-test-slave-6k22"
server_key: "3211aa84c731803ee351bcfd956493ef"
server_key_iv: "2c189adb2c13125ed8b38894cd6a8702"
server_key_version: "encryptionkey@0"
I20260807 18:37:21.541409 422 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786127579900523-11321-0/minicluster-data/ts-0/wal/instance:
uuid: "919e53b0c66843af85ad640da2d6bcb2"
format_stamp: "Formatted at 2026-08-07 18:37:21 on dist-test-slave-6k22"
server_key: "3211aa84c731803ee351bcfd956493ef"
server_key_iv: "2c189adb2c13125ed8b38894cd6a8702"
server_key_version: "encryptionkey@0"
I20260807 18:37:21.545209 422 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 18:37:21.547662 437 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:37:21.548825 422 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 18:37:21.548964 422 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786127579900523-11321-0/minicluster-data/ts-0/data,/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786127579900523-11321-0/minicluster-data/ts-0/wal
uuid: "919e53b0c66843af85ad640da2d6bcb2"
format_stamp: "Formatted at 2026-08-07 18:37:21 on dist-test-slave-6k22"
server_key: "3211aa84c731803ee351bcfd956493ef"
server_key_iv: "2c189adb2c13125ed8b38894cd6a8702"
server_key_version: "encryptionkey@0"
I20260807 18:37:21.549074 422 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786127579900523-11321-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786127579900523-11321-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786127579900523-11321-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:37:21.564391 422 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:37:21.565042 422 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:37:21.565238 422 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:37:21.565824 422 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:37:21.566810 422 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:37:21.566883 422 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:37:21.566959 422 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:37:21.567003 422 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:37:21.575820 422 rpc_server.cc:307] RPC server started. Bound to: 127.11.14.65:38027
I20260807 18:37:21.575863 555 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.14.65:38027 every 8 connection(s)
I20260807 18:37:21.576748 422 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786127579900523-11321-0/minicluster-data/ts-0/data/info.pb
I20260807 18:37:21.579471 11321 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkz56XN/build/debug/bin/kudu as pid 422
I20260807 18:37:21.579636 11321 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786127579900523-11321-0/minicluster-data/ts-0/wal/instance
I20260807 18:37:21.580017 11321 external_mini_cluster.cc:1428] Setting key 183b80aeed1baa14c97b96d7bf4eb9c5
I20260807 18:37:21.582373 11321 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkz56XN/build/debug/bin/kudu
/tmp/dist-test-taskkz56XN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786127579900523-11321-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786127579900523-11321-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786127579900523-11321-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786127579900523-11321-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.14.66:0
--local_ip_for_outbound_sockets=127.11.14.66
--webserver_interface=127.11.14.66
--webserver_port=0
--tserver_master_addrs=127.11.14.126:33763
--builtin_ntp_servers=127.11.14.84:41225
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 18:37:21.587495 399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:21.578585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.65:51261 (local address 127.11.14.126:33763)
0807 18:37:21.578741 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0807 18:37:21.578745 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:37:21.580450 (+ 1705us) server_negotiation.cc:407] Connection header received
0807 18:37:21.582162 (+ 1712us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:37:21.582165 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:37:21.582211 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:37:21.582279 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:37:21.583507 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:21.584093 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:21.585261 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:21.585645 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:21.585765 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:21.586886 (+ 1121us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:37:21.586893 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:37:21.586899 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:37:21.586919 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:37:21.586947 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:37:21.586963 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:37:21.587057 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0807 18:37:21.587206 (+ 149us) server_negotiation.cc:299] Negotiation successful
0807 18:37:21.587285 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260807 18:37:21.588341 559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:21.579040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:33763 (local address 127.11.14.65:51261)
0807 18:37:21.580240 (+ 1200us) negotiation.cc:107] Waiting for socket to connect
0807 18:37:21.580297 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0807 18:37:21.581926 (+ 1629us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:37:21.582475 (+ 549us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:37:21.582487 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:37:21.582606 (+ 119us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:37:21.583346 (+ 740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:37:21.583369 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:37:21.584226 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:37:21.584230 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:37:21.585129 (+ 899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:37:21.585138 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:37:21.585807 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:37:21.585811 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:37:21.585960 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:21.586602 (+ 642us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 18:37:21.586604 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:37:21.586649 (+ 45us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 18:37:21.586714 (+ 65us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:37:21.587060 (+ 346us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:37:21.587068 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:37:21.587072 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 18:37:21.587168 (+ 96us) client_negotiation.cc:240] Negotiation successful
0807 18:37:21.587467 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":373,"thread_start_us":150,"threads_started":1}
I20260807 18:37:21.589640 556 heartbeater.cc:347] Connected to a master server at 127.11.14.126:33763
I20260807 18:37:21.589927 556 heartbeater.cc:464] Registering TS with master...
I20260807 18:37:21.590605 556 heartbeater.cc:511] Master 127.11.14.126:33763 requested a full tablet report, sending...
I20260807 18:37:21.592293 355 ts_manager.cc:194] Registered new tserver with Master: 919e53b0c66843af85ad640da2d6bcb2 (127.11.14.65:38027)
I20260807 18:37:21.593427 355 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.14.65:51261
W20260807 18:37:21.701850 560 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:37:21.702090 560 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:37:21.702126 560 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:37:21.706125 560 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:37:21.706173 560 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:37:21.706274 560 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.14.66
I20260807 18:37:21.711114 560 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.14.84:41225
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786127579900523-11321-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786127579900523-11321-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.14.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786127579900523-11321-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.14.66
--webserver_port=0
--tserver_master_addrs=127.11.14.126:33763
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.560
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.14.66
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786127579900523-11321-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:37:21.712262 560 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:37:21.713606 560 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:37:21.723529 566 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:37:21.723510 570 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:37:21.723634 568 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:37:21.723768 560 server_base.cc:1034] running on GCE node
I20260807 18:37:21.724287 560 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:37:21.724992 560 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:37:21.726188 560 hybrid_clock.cc:648] HybridClock initialized: now 1786127841726162 us; error 38 us; skew 500 ppm
I20260807 18:37:21.728122 560 webserver.cc:460] Webserver started at http://127.11.14.66:39673/ using document root <none> and password file <none>
I20260807 18:37:21.728713 560 fs_manager.cc:359] Metadata directory not provided
I20260807 18:37:21.728814 560 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:37:21.729060 560 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:37:21.731240 560 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786127579900523-11321-0/minicluster-data/ts-1/data/instance:
uuid: "59783c79ddaf4e7aaa2469370832f90c"
format_stamp: "Formatted at 2026-08-07 18:37:21 on dist-test-slave-6k22"
server_key: "deb2b0f1787dd53e23af3d36c914a880"
server_key_iv: "60ba75faec0d8b0e9101727e3dcc7f7c"
server_key_version: "encryptionkey@0"
I20260807 18:37:21.731838 560 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786127579900523-11321-0/minicluster-data/ts-1/wal/instance:
uuid: "59783c79ddaf4e7aaa2469370832f90c"
format_stamp: "Formatted at 2026-08-07 18:37:21 on dist-test-slave-6k22"
server_key: "deb2b0f1787dd53e23af3d36c914a880"
server_key_iv: "60ba75faec0d8b0e9101727e3dcc7f7c"
server_key_version: "encryptionkey@0"
I20260807 18:37:21.735522 560 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260807 18:37:21.738082 577 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:37:21.739225 560 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 18:37:21.739363 560 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786127579900523-11321-0/minicluster-data/ts-1/data,/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786127579900523-11321-0/minicluster-data/ts-1/wal
uuid: "59783c79ddaf4e7aaa2469370832f90c"
format_stamp: "Formatted at 2026-08-07 18:37:21 on dist-test-slave-6k22"
server_key: "deb2b0f1787dd53e23af3d36c914a880"
server_key_iv: "60ba75faec0d8b0e9101727e3dcc7f7c"
server_key_version: "encryptionkey@0"
I20260807 18:37:21.739476 560 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786127579900523-11321-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786127579900523-11321-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786127579900523-11321-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:37:21.752969 560 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:37:21.753628 560 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:37:21.753827 560 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:37:21.754412 560 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:37:21.755349 560 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:37:21.755403 560 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:37:21.755472 560 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:37:21.755514 560 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:37:21.764889 560 rpc_server.cc:307] RPC server started. Bound to: 127.11.14.66:34777
I20260807 18:37:21.764925 705 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.14.66:34777 every 8 connection(s)
I20260807 18:37:21.765816 560 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786127579900523-11321-0/minicluster-data/ts-1/data/info.pb
I20260807 18:37:21.770219 11321 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkz56XN/build/debug/bin/kudu as pid 560
I20260807 18:37:21.770378 11321 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786127579900523-11321-0/minicluster-data/ts-1/wal/instance
I20260807 18:37:21.770713 11321 external_mini_cluster.cc:1428] Setting key f4989adb5257ff140985171ce33e82aa
I20260807 18:37:21.772848 11321 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkz56XN/build/debug/bin/kudu
/tmp/dist-test-taskkz56XN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786127579900523-11321-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786127579900523-11321-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786127579900523-11321-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786127579900523-11321-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.14.67:0
--local_ip_for_outbound_sockets=127.11.14.67
--webserver_interface=127.11.14.67
--webserver_port=0
--tserver_master_addrs=127.11.14.126:33763
--builtin_ntp_servers=127.11.14.84:41225
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 18:37:21.775724 399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:21.767728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.66:46639 (local address 127.11.14.126:33763)
0807 18:37:21.767879 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0807 18:37:21.767883 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:37:21.768937 (+ 1054us) server_negotiation.cc:407] Connection header received
0807 18:37:21.770352 (+ 1415us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:37:21.770357 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:37:21.770422 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:37:21.770529 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:37:21.771550 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:21.772217 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:21.773507 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:21.773892 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:21.774019 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:21.775113 (+ 1094us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:37:21.775120 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:37:21.775125 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:37:21.775143 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:37:21.775171 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:37:21.775183 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:37:21.775268 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0807 18:37:21.775425 (+ 157us) server_negotiation.cc:299] Negotiation successful
0807 18:37:21.775502 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260807 18:37:21.776549 708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:21.768143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:33763 (local address 127.11.14.66:46639)
0807 18:37:21.768781 (+ 638us) negotiation.cc:107] Waiting for socket to connect
0807 18:37:21.768831 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0807 18:37:21.770121 (+ 1290us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:37:21.770694 (+ 573us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:37:21.770701 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:37:21.770816 (+ 115us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:37:21.771441 (+ 625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:37:21.771459 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:37:21.772381 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:37:21.772384 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:37:21.773376 (+ 992us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:37:21.773387 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:37:21.774065 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:37:21.774068 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:37:21.774206 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:21.774851 (+ 645us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 18:37:21.774852 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:37:21.774892 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 18:37:21.774955 (+ 63us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:37:21.775276 (+ 321us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:37:21.775283 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:37:21.775288 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 18:37:21.775394 (+ 106us) client_negotiation.cc:240] Negotiation successful
0807 18:37:21.775679 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":119,"threads_started":1}
I20260807 18:37:21.777737 706 heartbeater.cc:347] Connected to a master server at 127.11.14.126:33763
I20260807 18:37:21.777966 706 heartbeater.cc:464] Registering TS with master...
I20260807 18:37:21.778491 706 heartbeater.cc:511] Master 127.11.14.126:33763 requested a full tablet report, sending...
I20260807 18:37:21.779522 355 ts_manager.cc:194] Registered new tserver with Master: 59783c79ddaf4e7aaa2469370832f90c (127.11.14.66:34777)
I20260807 18:37:21.780122 355 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.14.66:46639
W20260807 18:37:21.892153 709 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:37:21.892406 709 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:37:21.892444 709 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:37:21.896461 709 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:37:21.896509 709 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:37:21.896589 709 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.14.67
I20260807 18:37:21.901329 709 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.14.84:41225
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786127579900523-11321-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786127579900523-11321-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.14.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786127579900523-11321-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.14.67
--webserver_port=0
--tserver_master_addrs=127.11.14.126:33763
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.709
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.14.67
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786127579900523-11321-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:37:21.902360 709 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:37:21.903785 709 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:37:21.913347 715 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:37:21.913506 709 server_base.cc:1034] running on GCE node
W20260807 18:37:21.913336 719 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:37:21.913336 716 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:37:21.914058 709 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:37:21.914734 709 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:37:21.915916 709 hybrid_clock.cc:648] HybridClock initialized: now 1786127841915899 us; error 28 us; skew 500 ppm
I20260807 18:37:21.917701 709 webserver.cc:460] Webserver started at http://127.11.14.67:45463/ using document root <none> and password file <none>
I20260807 18:37:21.918294 709 fs_manager.cc:359] Metadata directory not provided
I20260807 18:37:21.918417 709 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:37:21.918653 709 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:37:21.920786 709 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786127579900523-11321-0/minicluster-data/ts-2/data/instance:
uuid: "c79e6cecedd34922a54bc7f8962a48dd"
format_stamp: "Formatted at 2026-08-07 18:37:21 on dist-test-slave-6k22"
server_key: "16c2de7f7d0a69b220ce9c811c829e7b"
server_key_iv: "19b85d8f238743b4c05deea37eaaea1d"
server_key_version: "encryptionkey@0"
I20260807 18:37:21.921360 709 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786127579900523-11321-0/minicluster-data/ts-2/wal/instance:
uuid: "c79e6cecedd34922a54bc7f8962a48dd"
format_stamp: "Formatted at 2026-08-07 18:37:21 on dist-test-slave-6k22"
server_key: "16c2de7f7d0a69b220ce9c811c829e7b"
server_key_iv: "19b85d8f238743b4c05deea37eaaea1d"
server_key_version: "encryptionkey@0"
I20260807 18:37:21.925087 709 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.004s
I20260807 18:37:21.927639 726 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:37:21.928907 709 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 18:37:21.929057 709 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786127579900523-11321-0/minicluster-data/ts-2/data,/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786127579900523-11321-0/minicluster-data/ts-2/wal
uuid: "c79e6cecedd34922a54bc7f8962a48dd"
format_stamp: "Formatted at 2026-08-07 18:37:21 on dist-test-slave-6k22"
server_key: "16c2de7f7d0a69b220ce9c811c829e7b"
server_key_iv: "19b85d8f238743b4c05deea37eaaea1d"
server_key_version: "encryptionkey@0"
I20260807 18:37:21.929179 709 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786127579900523-11321-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786127579900523-11321-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786127579900523-11321-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:37:21.942791 709 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:37:21.943495 709 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:37:21.943715 709 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:37:21.944329 709 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:37:21.945297 709 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:37:21.945379 709 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:37:21.945456 709 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:37:21.945510 709 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:37:21.954627 709 rpc_server.cc:307] RPC server started. Bound to: 127.11.14.67:39083
I20260807 18:37:21.954680 853 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.14.67:39083 every 8 connection(s)
I20260807 18:37:21.955646 709 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786127579900523-11321-0/minicluster-data/ts-2/data/info.pb
I20260807 18:37:21.960688 11321 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkz56XN/build/debug/bin/kudu as pid 709
I20260807 18:37:21.960824 11321 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786127579900523-11321-0/minicluster-data/ts-2/wal/instance
I20260807 18:37:21.961179 11321 external_mini_cluster.cc:1428] Setting key 3ce8f455572043980ae4b6ab36a8b451
I20260807 18:37:21.966022 399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:21.957842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.67:42635 (local address 127.11.14.126:33763)
0807 18:37:21.958002 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0807 18:37:21.958008 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:37:21.959370 (+ 1362us) server_negotiation.cc:407] Connection header received
0807 18:37:21.960945 (+ 1575us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:37:21.960950 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:37:21.961013 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:37:21.961107 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:37:21.962207 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:21.962931 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:21.964282 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:21.964502 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:21.964587 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:21.965618 (+ 1031us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:37:21.965621 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:37:21.965623 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:37:21.965636 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:37:21.965653 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:37:21.965661 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:37:21.965720 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0807 18:37:21.965862 (+ 142us) server_negotiation.cc:299] Negotiation successful
0807 18:37:21.965912 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260807 18:37:21.966969 856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:21.958362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:33763 (local address 127.11.14.67:42635)
0807 18:37:21.959211 (+ 849us) negotiation.cc:107] Waiting for socket to connect
0807 18:37:21.959270 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0807 18:37:21.960689 (+ 1419us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:37:21.961321 (+ 632us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:37:21.961329 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:37:21.961417 (+ 88us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:37:21.962036 (+ 619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:37:21.962058 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:37:21.963068 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:37:21.963071 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:37:21.964170 (+ 1099us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:37:21.964180 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:37:21.964650 (+ 470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:37:21.964654 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:37:21.964806 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:21.965373 (+ 567us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 18:37:21.965374 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:37:21.965415 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 18:37:21.965480 (+ 65us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:37:21.965737 (+ 257us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:37:21.965744 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:37:21.965749 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 18:37:21.965844 (+ 95us) client_negotiation.cc:240] Negotiation successful
0807 18:37:21.966103 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":341,"thread_start_us":117,"threads_started":1}
I20260807 18:37:21.968124 854 heartbeater.cc:347] Connected to a master server at 127.11.14.126:33763
I20260807 18:37:21.968385 854 heartbeater.cc:464] Registering TS with master...
I20260807 18:37:21.968926 854 heartbeater.cc:511] Master 127.11.14.126:33763 requested a full tablet report, sending...
I20260807 18:37:21.969944 355 ts_manager.cc:194] Registered new tserver with Master: c79e6cecedd34922a54bc7f8962a48dd (127.11.14.67:39083)
I20260807 18:37:21.970520 355 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.14.67:42635
I20260807 18:37:21.975405 11321 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 18:37:21.981418 399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:21.977653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38528 (local address 127.11.14.126:33763)
0807 18:37:21.977780 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0807 18:37:21.977784 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:37:21.978037 (+ 253us) server_negotiation.cc:407] Connection header received
0807 18:37:21.978158 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:37:21.978160 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:37:21.978199 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:37:21.978257 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:37:21.978510 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:21.978996 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:21.979995 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:21.980197 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:21.980279 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:21.981080 (+ 801us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:37:21.981083 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:37:21.981084 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:37:21.981096 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:37:21.981113 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:37:21.981121 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:37:21.981176 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0807 18:37:21.981259 (+ 83us) server_negotiation.cc:299] Negotiation successful
0807 18:37:21.981312 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260807 18:37:21.984244 355 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38528:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 18:37:21.987217 355 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 18:37:22.002267 867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:21.996457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.66:34777 (local address 127.0.0.1:42314)
0807 18:37:21.997816 (+ 1359us) negotiation.cc:107] Waiting for socket to connect
0807 18:37:21.997854 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0807 18:37:21.997956 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:37:21.998408 (+ 452us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:37:21.998411 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:37:21.998430 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:37:21.998544 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:37:21.998551 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:37:21.999892 (+ 1341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:37:21.999897 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:37:22.000693 (+ 796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:37:22.000702 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:37:22.001716 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:37:22.001722 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:37:22.001837 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:22.001874 (+ 37us) client_negotiation.cc:769] Sending connection context
0807 18:37:22.001989 (+ 115us) client_negotiation.cc:240] Negotiation successful
0807 18:37:22.002101 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1299,"mutex_wait_us":1,"thread_start_us":126,"threads_started":1}
I20260807 18:37:22.002605 871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:21.996384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47078 (local address 127.11.14.65:38027)
0807 18:37:21.997164 (+ 780us) server_negotiation.cc:206] Beginning negotiation
0807 18:37:21.997185 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0807 18:37:21.997211 (+ 26us) server_negotiation.cc:407] Connection header received
0807 18:37:21.997289 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:37:21.997294 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:37:21.997478 (+ 184us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:37:21.997640 (+ 162us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:37:21.998265 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:21.999373 (+ 1108us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:22.001109 (+ 1736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:22.001645 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:22.001748 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:22.001938 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0807 18:37:22.002337 (+ 399us) server_negotiation.cc:299] Negotiation successful
0807 18:37:22.002456 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":686,"thread_start_us":182,"threads_started":1}
I20260807 18:37:22.002660 868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:21.995689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.67:39083 (local address 127.0.0.1:33900)
0807 18:37:21.996923 (+ 1234us) negotiation.cc:107] Waiting for socket to connect
0807 18:37:21.996975 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0807 18:37:21.997130 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:37:21.997636 (+ 506us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:37:21.997642 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:37:21.997674 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:37:21.997876 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:37:21.997883 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:37:21.999348 (+ 1465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:37:21.999352 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:37:22.000958 (+ 1606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:37:22.000969 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:37:22.002206 (+ 1237us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:37:22.002210 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:37:22.002319 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:22.002344 (+ 25us) client_negotiation.cc:769] Sending connection context
0807 18:37:22.002418 (+ 74us) client_negotiation.cc:240] Negotiation successful
0807 18:37:22.002504 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1043,"mutex_wait_us":112,"spinlock_wait_cycles":10368,"thread_start_us":99,"threads_started":1}
I20260807 18:37:22.002990 869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:21.995977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33900 (local address 127.11.14.67:39083)
0807 18:37:21.996945 (+ 968us) server_negotiation.cc:206] Beginning negotiation
0807 18:37:21.996958 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 18:37:21.997263 (+ 305us) server_negotiation.cc:407] Connection header received
0807 18:37:21.997335 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:37:21.997338 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:37:21.997494 (+ 156us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:37:21.997644 (+ 150us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:37:21.997999 (+ 355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:21.999206 (+ 1207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:22.001086 (+ 1880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:22.001830 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:22.001938 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:22.002135 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0807 18:37:22.002755 (+ 620us) server_negotiation.cc:299] Negotiation successful
0807 18:37:22.002839 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":873,"thread_start_us":130,"threads_started":1}
I20260807 18:37:22.002990 873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:21.996668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42314 (local address 127.11.14.66:34777)
0807 18:37:21.997124 (+ 456us) server_negotiation.cc:206] Beginning negotiation
0807 18:37:21.997136 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 18:37:21.997945 (+ 809us) server_negotiation.cc:407] Connection header received
0807 18:37:21.998067 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:37:21.998072 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:37:21.998256 (+ 184us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:37:21.998431 (+ 175us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:37:21.998662 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:21.999784 (+ 1122us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:22.000832 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:22.001553 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:22.001667 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:22.001884 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0807 18:37:22.002717 (+ 833us) server_negotiation.cc:299] Negotiation successful
0807 18:37:22.002830 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":362,"thread_start_us":155,"threads_started":1}
I20260807 18:37:22.002700 872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:21.995765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.65:38027 (local address 127.0.0.1:47078)
0807 18:37:21.996922 (+ 1157us) negotiation.cc:107] Waiting for socket to connect
0807 18:37:21.996976 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0807 18:37:21.997129 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:37:21.997638 (+ 509us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:37:21.997643 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:37:21.997675 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:37:21.998144 (+ 469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:37:21.998151 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:37:21.999492 (+ 1341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:37:21.999495 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:37:22.000958 (+ 1463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:37:22.000969 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:37:22.002001 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:37:22.002004 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:37:22.002085 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:22.002105 (+ 20us) client_negotiation.cc:769] Sending connection context
0807 18:37:22.002170 (+ 65us) client_negotiation.cc:240] Negotiation successful
0807 18:37:22.002222 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":998,"mutex_wait_us":463,"spinlock_wait_cycles":7808,"thread_start_us":103,"threads_started":1}
I20260807 18:37:22.006127 780 tablet_service.cc:1467] Processing CreateTablet for tablet 3f6e74af6f224208894ddab25cfd6c86 (DEFAULT_TABLE table=test-table [id=0d6d2b1a18de4524b54dd8b143f38ac4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:37:22.006127 487 tablet_service.cc:1467] Processing CreateTablet for tablet 3f6e74af6f224208894ddab25cfd6c86 (DEFAULT_TABLE table=test-table [id=0d6d2b1a18de4524b54dd8b143f38ac4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:37:22.007319 487 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3f6e74af6f224208894ddab25cfd6c86. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:37:22.007319 780 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3f6e74af6f224208894ddab25cfd6c86. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:37:22.009421 630 tablet_service.cc:1467] Processing CreateTablet for tablet 3f6e74af6f224208894ddab25cfd6c86 (DEFAULT_TABLE table=test-table [id=0d6d2b1a18de4524b54dd8b143f38ac4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:37:22.010494 630 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3f6e74af6f224208894ddab25cfd6c86. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:37:22.015012 874 tablet_bootstrap.cc:492] T 3f6e74af6f224208894ddab25cfd6c86 P c79e6cecedd34922a54bc7f8962a48dd: Bootstrap starting.
I20260807 18:37:22.015434 875 tablet_bootstrap.cc:492] T 3f6e74af6f224208894ddab25cfd6c86 P 919e53b0c66843af85ad640da2d6bcb2: Bootstrap starting.
I20260807 18:37:22.016471 876 tablet_bootstrap.cc:492] T 3f6e74af6f224208894ddab25cfd6c86 P 59783c79ddaf4e7aaa2469370832f90c: Bootstrap starting.
I20260807 18:37:22.018381 876 tablet_bootstrap.cc:654] T 3f6e74af6f224208894ddab25cfd6c86 P 59783c79ddaf4e7aaa2469370832f90c: Neither blocks nor log segments found. Creating new log.
I20260807 18:37:22.018401 874 tablet_bootstrap.cc:654] T 3f6e74af6f224208894ddab25cfd6c86 P c79e6cecedd34922a54bc7f8962a48dd: Neither blocks nor log segments found. Creating new log.
I20260807 18:37:22.018708 875 tablet_bootstrap.cc:654] T 3f6e74af6f224208894ddab25cfd6c86 P 919e53b0c66843af85ad640da2d6bcb2: Neither blocks nor log segments found. Creating new log.
I20260807 18:37:22.019198 876 log.cc:824] T 3f6e74af6f224208894ddab25cfd6c86 P 59783c79ddaf4e7aaa2469370832f90c: Log is configured to *not* fsync() on all Append() calls
I20260807 18:37:22.019259 874 log.cc:824] T 3f6e74af6f224208894ddab25cfd6c86 P c79e6cecedd34922a54bc7f8962a48dd: Log is configured to *not* fsync() on all Append() calls
I20260807 18:37:22.019448 875 log.cc:824] T 3f6e74af6f224208894ddab25cfd6c86 P 919e53b0c66843af85ad640da2d6bcb2: Log is configured to *not* fsync() on all Append() calls
I20260807 18:37:22.021349 876 tablet_bootstrap.cc:492] T 3f6e74af6f224208894ddab25cfd6c86 P 59783c79ddaf4e7aaa2469370832f90c: No bootstrap required, opened a new log
I20260807 18:37:22.021559 874 tablet_bootstrap.cc:492] T 3f6e74af6f224208894ddab25cfd6c86 P c79e6cecedd34922a54bc7f8962a48dd: No bootstrap required, opened a new log
I20260807 18:37:22.021564 876 ts_tablet_manager.cc:1397] T 3f6e74af6f224208894ddab25cfd6c86 P 59783c79ddaf4e7aaa2469370832f90c: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.003s
I20260807 18:37:22.021749 874 ts_tablet_manager.cc:1397] T 3f6e74af6f224208894ddab25cfd6c86 P c79e6cecedd34922a54bc7f8962a48dd: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260807 18:37:22.021927 875 tablet_bootstrap.cc:492] T 3f6e74af6f224208894ddab25cfd6c86 P 919e53b0c66843af85ad640da2d6bcb2: No bootstrap required, opened a new log
I20260807 18:37:22.022118 875 ts_tablet_manager.cc:1397] T 3f6e74af6f224208894ddab25cfd6c86 P 919e53b0c66843af85ad640da2d6bcb2: Time spent bootstrapping tablet: real 0.007s user 0.003s sys 0.000s
I20260807 18:37:22.025192 876 raft_consensus.cc:348] T 3f6e74af6f224208894ddab25cfd6c86 P 59783c79ddaf4e7aaa2469370832f90c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c79e6cecedd34922a54bc7f8962a48dd" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 39083 } } peers { permanent_uuid: "919e53b0c66843af85ad640da2d6bcb2" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 38027 } } peers { permanent_uuid: "59783c79ddaf4e7aaa2469370832f90c" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 34777 } }
I20260807 18:37:22.025444 876 raft_consensus.cc:374] T 3f6e74af6f224208894ddab25cfd6c86 P 59783c79ddaf4e7aaa2469370832f90c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:37:22.025517 876 raft_consensus.cc:729] T 3f6e74af6f224208894ddab25cfd6c86 P 59783c79ddaf4e7aaa2469370832f90c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 59783c79ddaf4e7aaa2469370832f90c, State: Initialized, Role: FOLLOWER
I20260807 18:37:22.025962 876 consensus_queue.cc:260] T 3f6e74af6f224208894ddab25cfd6c86 P 59783c79ddaf4e7aaa2469370832f90c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c79e6cecedd34922a54bc7f8962a48dd" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 39083 } } peers { permanent_uuid: "919e53b0c66843af85ad640da2d6bcb2" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 38027 } } peers { permanent_uuid: "59783c79ddaf4e7aaa2469370832f90c" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 34777 } }
I20260807 18:37:22.025995 874 raft_consensus.cc:348] T 3f6e74af6f224208894ddab25cfd6c86 P c79e6cecedd34922a54bc7f8962a48dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c79e6cecedd34922a54bc7f8962a48dd" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 39083 } } peers { permanent_uuid: "919e53b0c66843af85ad640da2d6bcb2" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 38027 } } peers { permanent_uuid: "59783c79ddaf4e7aaa2469370832f90c" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 34777 } }
I20260807 18:37:22.026243 874 raft_consensus.cc:374] T 3f6e74af6f224208894ddab25cfd6c86 P c79e6cecedd34922a54bc7f8962a48dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:37:22.026153 875 raft_consensus.cc:348] T 3f6e74af6f224208894ddab25cfd6c86 P 919e53b0c66843af85ad640da2d6bcb2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c79e6cecedd34922a54bc7f8962a48dd" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 39083 } } peers { permanent_uuid: "919e53b0c66843af85ad640da2d6bcb2" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 38027 } } peers { permanent_uuid: "59783c79ddaf4e7aaa2469370832f90c" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 34777 } }
I20260807 18:37:22.026363 874 raft_consensus.cc:729] T 3f6e74af6f224208894ddab25cfd6c86 P c79e6cecedd34922a54bc7f8962a48dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c79e6cecedd34922a54bc7f8962a48dd, State: Initialized, Role: FOLLOWER
I20260807 18:37:22.026468 875 raft_consensus.cc:374] T 3f6e74af6f224208894ddab25cfd6c86 P 919e53b0c66843af85ad640da2d6bcb2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:37:22.026561 875 raft_consensus.cc:729] T 3f6e74af6f224208894ddab25cfd6c86 P 919e53b0c66843af85ad640da2d6bcb2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 919e53b0c66843af85ad640da2d6bcb2, State: Initialized, Role: FOLLOWER
I20260807 18:37:22.026626 706 heartbeater.cc:503] Master 127.11.14.126:33763 was elected leader, sending a full tablet report...
I20260807 18:37:22.026825 874 consensus_queue.cc:260] T 3f6e74af6f224208894ddab25cfd6c86 P c79e6cecedd34922a54bc7f8962a48dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c79e6cecedd34922a54bc7f8962a48dd" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 39083 } } peers { permanent_uuid: "919e53b0c66843af85ad640da2d6bcb2" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 38027 } } peers { permanent_uuid: "59783c79ddaf4e7aaa2469370832f90c" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 34777 } }
I20260807 18:37:22.026958 876 ts_tablet_manager.cc:1428] T 3f6e74af6f224208894ddab25cfd6c86 P 59783c79ddaf4e7aaa2469370832f90c: Time spent starting tablet: real 0.005s user 0.004s sys 0.002s
I20260807 18:37:22.027086 875 consensus_queue.cc:260] T 3f6e74af6f224208894ddab25cfd6c86 P 919e53b0c66843af85ad640da2d6bcb2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c79e6cecedd34922a54bc7f8962a48dd" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 39083 } } peers { permanent_uuid: "919e53b0c66843af85ad640da2d6bcb2" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 38027 } } peers { permanent_uuid: "59783c79ddaf4e7aaa2469370832f90c" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 34777 } }
I20260807 18:37:22.027863 874 ts_tablet_manager.cc:1428] T 3f6e74af6f224208894ddab25cfd6c86 P c79e6cecedd34922a54bc7f8962a48dd: Time spent starting tablet: real 0.006s user 0.002s sys 0.005s
I20260807 18:37:22.027859 556 heartbeater.cc:503] Master 127.11.14.126:33763 was elected leader, sending a full tablet report...
I20260807 18:37:22.028044 875 ts_tablet_manager.cc:1428] T 3f6e74af6f224208894ddab25cfd6c86 P 919e53b0c66843af85ad640da2d6bcb2: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260807 18:37:22.028234 854 heartbeater.cc:503] Master 127.11.14.126:33763 was elected leader, sending a full tablet report...
W20260807 18:37:22.078056 558 tablet.cc:2369] T 3f6e74af6f224208894ddab25cfd6c86 P 919e53b0c66843af85ad640da2d6bcb2: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 18:37:22.206743 855 tablet.cc:2369] T 3f6e74af6f224208894ddab25cfd6c86 P c79e6cecedd34922a54bc7f8962a48dd: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 18:37:22.267069 707 tablet.cc:2369] T 3f6e74af6f224208894ddab25cfd6c86 P 59783c79ddaf4e7aaa2469370832f90c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:37:22.314086 883 raft_consensus.cc:482] T 3f6e74af6f224208894ddab25cfd6c86 P 919e53b0c66843af85ad640da2d6bcb2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:37:22.314370 883 raft_consensus.cc:504] T 3f6e74af6f224208894ddab25cfd6c86 P 919e53b0c66843af85ad640da2d6bcb2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c79e6cecedd34922a54bc7f8962a48dd" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 39083 } } peers { permanent_uuid: "919e53b0c66843af85ad640da2d6bcb2" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 38027 } } peers { permanent_uuid: "59783c79ddaf4e7aaa2469370832f90c" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 34777 } }
I20260807 18:37:22.315954 883 leader_election.cc:288] T 3f6e74af6f224208894ddab25cfd6c86 P 919e53b0c66843af85ad640da2d6bcb2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c79e6cecedd34922a54bc7f8962a48dd (127.11.14.67:39083), 59783c79ddaf4e7aaa2469370832f90c (127.11.14.66:34777)
I20260807 18:37:22.316975 881 raft_consensus.cc:482] T 3f6e74af6f224208894ddab25cfd6c86 P 59783c79ddaf4e7aaa2469370832f90c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:37:22.317193 881 raft_consensus.cc:504] T 3f6e74af6f224208894ddab25cfd6c86 P 59783c79ddaf4e7aaa2469370832f90c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c79e6cecedd34922a54bc7f8962a48dd" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 39083 } } peers { permanent_uuid: "919e53b0c66843af85ad640da2d6bcb2" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 38027 } } peers { permanent_uuid: "59783c79ddaf4e7aaa2469370832f90c" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 34777 } }
I20260807 18:37:22.319139 881 leader_election.cc:288] T 3f6e74af6f224208894ddab25cfd6c86 P 59783c79ddaf4e7aaa2469370832f90c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c79e6cecedd34922a54bc7f8962a48dd (127.11.14.67:39083), 919e53b0c66843af85ad640da2d6bcb2 (127.11.14.65:38027)
I20260807 18:37:22.321779 873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:22.316393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.65:58617 (local address 127.11.14.66:34777)
0807 18:37:22.316543 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0807 18:37:22.316548 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:37:22.316834 (+ 286us) server_negotiation.cc:407] Connection header received
0807 18:37:22.316980 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:37:22.316984 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:37:22.317044 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:37:22.317122 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:37:22.317594 (+ 472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:22.318243 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:22.320322 (+ 2079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:22.320966 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:22.321079 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:22.321141 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0807 18:37:22.321541 (+ 400us) server_negotiation.cc:299] Negotiation successful
0807 18:37:22.321634 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260807 18:37:22.321807 869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:22.316275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.65:54519 (local address 127.11.14.67:39083)
0807 18:37:22.316427 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0807 18:37:22.316432 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:37:22.316713 (+ 281us) server_negotiation.cc:407] Connection header received
0807 18:37:22.316981 (+ 268us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:37:22.316984 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:37:22.317044 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:37:22.317103 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:37:22.317702 (+ 599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:22.318474 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:22.320225 (+ 1751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:22.320862 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:22.320953 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:22.321003 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0807 18:37:22.321587 (+ 584us) server_negotiation.cc:299] Negotiation successful
0807 18:37:22.321666 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260807 18:37:22.322232 885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:22.316172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.67:39083 (local address 127.11.14.65:54519)
0807 18:37:22.316570 (+ 398us) negotiation.cc:107] Waiting for socket to connect
0807 18:37:22.316595 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0807 18:37:22.316781 (+ 186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:37:22.317288 (+ 507us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:37:22.317290 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:37:22.317348 (+ 58us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:37:22.317511 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:37:22.317518 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:37:22.318619 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:37:22.318626 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:37:22.320085 (+ 1459us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:37:22.320097 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:37:22.321329 (+ 1232us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:37:22.321332 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:37:22.321439 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:22.321461 (+ 22us) client_negotiation.cc:769] Sending connection context
0807 18:37:22.321992 (+ 531us) client_negotiation.cc:240] Negotiation successful
0807 18:37:22.322070 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":324,"mutex_wait_us":6,"spinlock_wait_cycles":896,"thread_start_us":127,"threads_started":1}
I20260807 18:37:22.322376 884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:22.316254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.66:34777 (local address 127.11.14.65:58617)
0807 18:37:22.316673 (+ 419us) negotiation.cc:107] Waiting for socket to connect
0807 18:37:22.316715 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 18:37:22.316860 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:37:22.317242 (+ 382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:37:22.317245 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:37:22.317265 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:37:22.317430 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:37:22.317436 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:37:22.318387 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:37:22.318391 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:37:22.320184 (+ 1793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:37:22.320193 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:37:22.321258 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:37:22.321261 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:37:22.321381 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:22.321405 (+ 24us) client_negotiation.cc:769] Sending connection context
0807 18:37:22.322060 (+ 655us) client_negotiation.cc:240] Negotiation successful
0807 18:37:22.322163 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":360,"mutex_wait_us":433,"spinlock_wait_cycles":29312,"thread_start_us":90,"threads_started":1}
I20260807 18:37:22.323062 804 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f6e74af6f224208894ddab25cfd6c86" candidate_uuid: "919e53b0c66843af85ad640da2d6bcb2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c79e6cecedd34922a54bc7f8962a48dd" is_pre_election: true
I20260807 18:37:22.323407 804 raft_consensus.cc:2454] T 3f6e74af6f224208894ddab25cfd6c86 P c79e6cecedd34922a54bc7f8962a48dd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 919e53b0c66843af85ad640da2d6bcb2 in term 0.
I20260807 18:37:22.323758 652 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f6e74af6f224208894ddab25cfd6c86" candidate_uuid: "919e53b0c66843af85ad640da2d6bcb2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "59783c79ddaf4e7aaa2469370832f90c" is_pre_election: true
I20260807 18:37:22.324023 652 raft_consensus.cc:2454] T 3f6e74af6f224208894ddab25cfd6c86 P 59783c79ddaf4e7aaa2469370832f90c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 919e53b0c66843af85ad640da2d6bcb2 in term 0.
I20260807 18:37:22.324092 889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:22.318980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.66:50103 (local address 127.11.14.67:39083)
0807 18:37:22.319850 (+ 870us) server_negotiation.cc:206] Beginning negotiation
0807 18:37:22.319858 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:37:22.319871 (+ 13us) server_negotiation.cc:407] Connection header received
0807 18:37:22.319911 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:37:22.319913 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:37:22.319952 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:37:22.320033 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:37:22.320443 (+ 410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:22.321137 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:22.322695 (+ 1558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:22.323264 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:22.323394 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:22.323449 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0807 18:37:22.323849 (+ 400us) server_negotiation.cc:299] Negotiation successful
0807 18:37:22.323925 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":799,"thread_start_us":144,"threads_started":1}
I20260807 18:37:22.324084 441 leader_election.cc:302] T 3f6e74af6f224208894ddab25cfd6c86 P 919e53b0c66843af85ad640da2d6bcb2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 919e53b0c66843af85ad640da2d6bcb2, c79e6cecedd34922a54bc7f8962a48dd; no voters:
I20260807 18:37:22.324330 871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:22.319071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.14.66:40047 (local address 127.11.14.65:38027)
0807 18:37:22.319172 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0807 18:37:22.319177 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:37:22.319588 (+ 411us) server_negotiation.cc:407] Connection header received
0807 18:37:22.319683 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:37:22.319687 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:37:22.319732 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:37:22.319812 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:37:22.320181 (+ 369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:22.321002 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:22.322922 (+ 1920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:22.323533 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:22.323693 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:22.323737 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0807 18:37:22.324120 (+ 383us) server_negotiation.cc:299] Negotiation successful
0807 18:37:22.324210 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260807 18:37:22.324560 888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:22.318677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.67:39083 (local address 127.11.14.66:50103)
0807 18:37:22.319014 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0807 18:37:22.319029 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 18:37:22.319140 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:37:22.320143 (+ 1003us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:37:22.320149 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:37:22.320174 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:37:22.320320 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:37:22.320328 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:37:22.321252 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:37:22.321257 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:37:22.322558 (+ 1301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:37:22.322574 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:37:22.323397 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:37:22.323401 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:37:22.323523 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:22.323551 (+ 28us) client_negotiation.cc:769] Sending connection context
0807 18:37:22.323659 (+ 108us) client_negotiation.cc:240] Negotiation successful
0807 18:37:22.323736 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":265,"mutex_wait_us":41,"thread_start_us":77,"threads_started":1}
I20260807 18:37:22.324683 883 raft_consensus.cc:2781] T 3f6e74af6f224208894ddab25cfd6c86 P 919e53b0c66843af85ad640da2d6bcb2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:37:22.324770 887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:22.318726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.65:38027 (local address 127.11.14.66:40047)
0807 18:37:22.319435 (+ 709us) negotiation.cc:107] Waiting for socket to connect
0807 18:37:22.319461 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0807 18:37:22.319585 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:37:22.319917 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:37:22.319921 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:37:22.319939 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:37:22.320050 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:37:22.320056 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:37:22.321148 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:37:22.321151 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:37:22.322768 (+ 1617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:37:22.322785 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:37:22.323855 (+ 1070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:37:22.323860 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:37:22.323959 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:22.323982 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 18:37:22.324441 (+ 459us) client_negotiation.cc:240] Negotiation successful
0807 18:37:22.324520 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":646,"mutex_wait_us":1,"thread_start_us":76,"threads_started":1}
I20260807 18:37:22.324796 883 raft_consensus.cc:482] T 3f6e74af6f224208894ddab25cfd6c86 P 919e53b0c66843af85ad640da2d6bcb2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:37:22.324860 883 raft_consensus.cc:3037] T 3f6e74af6f224208894ddab25cfd6c86 P 919e53b0c66843af85ad640da2d6bcb2 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:37:22.325139 804 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f6e74af6f224208894ddab25cfd6c86" candidate_uuid: "59783c79ddaf4e7aaa2469370832f90c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c79e6cecedd34922a54bc7f8962a48dd" is_pre_election: true
I20260807 18:37:22.325277 804 raft_consensus.cc:2454] T 3f6e74af6f224208894ddab25cfd6c86 P c79e6cecedd34922a54bc7f8962a48dd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 59783c79ddaf4e7aaa2469370832f90c in term 0.
I20260807 18:37:22.325321 508 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f6e74af6f224208894ddab25cfd6c86" candidate_uuid: "59783c79ddaf4e7aaa2469370832f90c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "919e53b0c66843af85ad640da2d6bcb2" is_pre_election: true
I20260807 18:37:22.325713 581 leader_election.cc:302] T 3f6e74af6f224208894ddab25cfd6c86 P 59783c79ddaf4e7aaa2469370832f90c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 59783c79ddaf4e7aaa2469370832f90c, c79e6cecedd34922a54bc7f8962a48dd; no voters:
I20260807 18:37:22.326022 881 raft_consensus.cc:2781] T 3f6e74af6f224208894ddab25cfd6c86 P 59783c79ddaf4e7aaa2469370832f90c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:37:22.326131 881 raft_consensus.cc:482] T 3f6e74af6f224208894ddab25cfd6c86 P 59783c79ddaf4e7aaa2469370832f90c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:37:22.326176 881 raft_consensus.cc:3037] T 3f6e74af6f224208894ddab25cfd6c86 P 59783c79ddaf4e7aaa2469370832f90c [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:37:22.326442 883 raft_consensus.cc:504] T 3f6e74af6f224208894ddab25cfd6c86 P 919e53b0c66843af85ad640da2d6bcb2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c79e6cecedd34922a54bc7f8962a48dd" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 39083 } } peers { permanent_uuid: "919e53b0c66843af85ad640da2d6bcb2" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 38027 } } peers { permanent_uuid: "59783c79ddaf4e7aaa2469370832f90c" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 34777 } }
I20260807 18:37:22.326840 508 raft_consensus.cc:2379] T 3f6e74af6f224208894ddab25cfd6c86 P 919e53b0c66843af85ad640da2d6bcb2 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 59783c79ddaf4e7aaa2469370832f90c in current term 1: Already voted for candidate 919e53b0c66843af85ad640da2d6bcb2 in this term.
I20260807 18:37:22.327203 652 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f6e74af6f224208894ddab25cfd6c86" candidate_uuid: "919e53b0c66843af85ad640da2d6bcb2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "59783c79ddaf4e7aaa2469370832f90c"
I20260807 18:37:22.327416 883 leader_election.cc:288] T 3f6e74af6f224208894ddab25cfd6c86 P 919e53b0c66843af85ad640da2d6bcb2 [CANDIDATE]: Term 1 election: Requested vote from peers c79e6cecedd34922a54bc7f8962a48dd (127.11.14.67:39083), 59783c79ddaf4e7aaa2469370832f90c (127.11.14.66:34777)
I20260807 18:37:22.327490 881 raft_consensus.cc:504] T 3f6e74af6f224208894ddab25cfd6c86 P 59783c79ddaf4e7aaa2469370832f90c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c79e6cecedd34922a54bc7f8962a48dd" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 39083 } } peers { permanent_uuid: "919e53b0c66843af85ad640da2d6bcb2" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 38027 } } peers { permanent_uuid: "59783c79ddaf4e7aaa2469370832f90c" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 34777 } }
I20260807 18:37:22.327657 804 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f6e74af6f224208894ddab25cfd6c86" candidate_uuid: "919e53b0c66843af85ad640da2d6bcb2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c79e6cecedd34922a54bc7f8962a48dd"
I20260807 18:37:22.327762 804 raft_consensus.cc:3037] T 3f6e74af6f224208894ddab25cfd6c86 P c79e6cecedd34922a54bc7f8962a48dd [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:37:22.327804 652 raft_consensus.cc:2379] T 3f6e74af6f224208894ddab25cfd6c86 P 59783c79ddaf4e7aaa2469370832f90c [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 919e53b0c66843af85ad640da2d6bcb2 in current term 1: Already voted for candidate 59783c79ddaf4e7aaa2469370832f90c in this term.
I20260807 18:37:22.327924 881 leader_election.cc:288] T 3f6e74af6f224208894ddab25cfd6c86 P 59783c79ddaf4e7aaa2469370832f90c [CANDIDATE]: Term 1 election: Requested vote from peers c79e6cecedd34922a54bc7f8962a48dd (127.11.14.67:39083), 919e53b0c66843af85ad640da2d6bcb2 (127.11.14.65:38027)
I20260807 18:37:22.328390 803 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f6e74af6f224208894ddab25cfd6c86" candidate_uuid: "59783c79ddaf4e7aaa2469370832f90c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c79e6cecedd34922a54bc7f8962a48dd"
I20260807 18:37:22.328948 804 raft_consensus.cc:2454] T 3f6e74af6f224208894ddab25cfd6c86 P c79e6cecedd34922a54bc7f8962a48dd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 919e53b0c66843af85ad640da2d6bcb2 in term 1.
I20260807 18:37:22.329555 508 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f6e74af6f224208894ddab25cfd6c86" candidate_uuid: "59783c79ddaf4e7aaa2469370832f90c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "919e53b0c66843af85ad640da2d6bcb2"
I20260807 18:37:22.329612 441 leader_election.cc:302] T 3f6e74af6f224208894ddab25cfd6c86 P 919e53b0c66843af85ad640da2d6bcb2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 919e53b0c66843af85ad640da2d6bcb2, c79e6cecedd34922a54bc7f8962a48dd; no voters: 59783c79ddaf4e7aaa2469370832f90c
I20260807 18:37:22.329725 508 raft_consensus.cc:2379] T 3f6e74af6f224208894ddab25cfd6c86 P 919e53b0c66843af85ad640da2d6bcb2 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 59783c79ddaf4e7aaa2469370832f90c in current term 1: Already voted for candidate 919e53b0c66843af85ad640da2d6bcb2 in this term.
I20260807 18:37:22.329942 883 raft_consensus.cc:2781] T 3f6e74af6f224208894ddab25cfd6c86 P 919e53b0c66843af85ad640da2d6bcb2 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:37:22.330137 578 leader_election.cc:302] T 3f6e74af6f224208894ddab25cfd6c86 P 59783c79ddaf4e7aaa2469370832f90c [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 59783c79ddaf4e7aaa2469370832f90c; no voters: 919e53b0c66843af85ad640da2d6bcb2, c79e6cecedd34922a54bc7f8962a48dd
I20260807 18:37:22.330191 883 raft_consensus.cc:686] T 3f6e74af6f224208894ddab25cfd6c86 P 919e53b0c66843af85ad640da2d6bcb2 [term 1 LEADER]: Becoming Leader. State: Replica: 919e53b0c66843af85ad640da2d6bcb2, State: Running, Role: LEADER
I20260807 18:37:22.330391 881 raft_consensus.cc:2726] T 3f6e74af6f224208894ddab25cfd6c86 P 59783c79ddaf4e7aaa2469370832f90c [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260807 18:37:22.330549 883 consensus_queue.cc:237] T 3f6e74af6f224208894ddab25cfd6c86 P 919e53b0c66843af85ad640da2d6bcb2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c79e6cecedd34922a54bc7f8962a48dd" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 39083 } } peers { permanent_uuid: "919e53b0c66843af85ad640da2d6bcb2" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 38027 } } peers { permanent_uuid: "59783c79ddaf4e7aaa2469370832f90c" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 34777 } }
I20260807 18:37:22.333928 354 catalog_manager.cc:5495] T 3f6e74af6f224208894ddab25cfd6c86 P 919e53b0c66843af85ad640da2d6bcb2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 919e53b0c66843af85ad640da2d6bcb2 (127.11.14.65). New cstate: current_term: 1 leader_uuid: "919e53b0c66843af85ad640da2d6bcb2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c79e6cecedd34922a54bc7f8962a48dd" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 39083 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "919e53b0c66843af85ad640da2d6bcb2" member_type: VOTER last_known_addr { host: "127.11.14.65" port: 38027 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "59783c79ddaf4e7aaa2469370832f90c" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 34777 } health_report { overall_health: UNKNOWN } } }
I20260807 18:37:22.364784 871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:22.361024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47090 (local address 127.11.14.65:38027)
0807 18:37:22.361169 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0807 18:37:22.361174 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:37:22.361190 (+ 16us) server_negotiation.cc:407] Connection header received
0807 18:37:22.361361 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:37:22.361364 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:37:22.361422 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:37:22.361533 (+ 111us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:37:22.361783 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:22.362310 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:22.363247 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:22.363472 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:22.363602 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:22.363932 (+ 330us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:37:22.364370 (+ 438us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:37:22.364506 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0807 18:37:22.364572 (+ 66us) server_negotiation.cc:299] Negotiation successful
0807 18:37:22.364633 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260807 18:37:22.370822 652 raft_consensus.cc:1261] T 3f6e74af6f224208894ddab25cfd6c86 P 59783c79ddaf4e7aaa2469370832f90c [term 1 FOLLOWER]: Refusing update from remote peer 919e53b0c66843af85ad640da2d6bcb2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:37:22.370815 804 raft_consensus.cc:1261] T 3f6e74af6f224208894ddab25cfd6c86 P c79e6cecedd34922a54bc7f8962a48dd [term 1 FOLLOWER]: Refusing update from remote peer 919e53b0c66843af85ad640da2d6bcb2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:37:22.371433 883 consensus_queue.cc:1035] T 3f6e74af6f224208894ddab25cfd6c86 P 919e53b0c66843af85ad640da2d6bcb2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "59783c79ddaf4e7aaa2469370832f90c" member_type: VOTER last_known_addr { host: "127.11.14.66" port: 34777 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:37:22.371785 890 consensus_queue.cc:1035] T 3f6e74af6f224208894ddab25cfd6c86 P 919e53b0c66843af85ad640da2d6bcb2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c79e6cecedd34922a54bc7f8962a48dd" member_type: VOTER last_known_addr { host: "127.11.14.67" port: 39083 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:37:22.389305 354 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38528:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:37:22.389501 354 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38528:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:37:22.392247 354 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 224a5b31ef3442d7a9cb3f8b42b80736: Sending DeleteTablet for 3 replicas of tablet 3f6e74af6f224208894ddab25cfd6c86
I20260807 18:37:22.393136 780 tablet_service.cc:1514] Processing DeleteTablet for tablet 3f6e74af6f224208894ddab25cfd6c86 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:37:22 UTC) from {username='slave'} at 127.0.0.1:33900
I20260807 18:37:22.393344 630 tablet_service.cc:1514] Processing DeleteTablet for tablet 3f6e74af6f224208894ddab25cfd6c86 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:37:22 UTC) from {username='slave'} at 127.0.0.1:42314
I20260807 18:37:22.393373 487 tablet_service.cc:1514] Processing DeleteTablet for tablet 3f6e74af6f224208894ddab25cfd6c86 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:37:22 UTC) from {username='slave'} at 127.0.0.1:47078
I20260807 18:37:22.393800 903 tablet_replica.cc:330] T 3f6e74af6f224208894ddab25cfd6c86 P 59783c79ddaf4e7aaa2469370832f90c: stopping tablet replica
I20260807 18:37:22.394030 902 tablet_replica.cc:330] T 3f6e74af6f224208894ddab25cfd6c86 P c79e6cecedd34922a54bc7f8962a48dd: stopping tablet replica
I20260807 18:37:22.394196 903 raft_consensus.cc:2229] T 3f6e74af6f224208894ddab25cfd6c86 P 59783c79ddaf4e7aaa2469370832f90c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:37:22.394212 11321 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkz56XN/build/debug/bin/kudu with pid 422
I20260807 18:37:22.394364 902 raft_consensus.cc:2229] T 3f6e74af6f224208894ddab25cfd6c86 P c79e6cecedd34922a54bc7f8962a48dd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:37:22.394589 903 raft_consensus.cc:2258] T 3f6e74af6f224208894ddab25cfd6c86 P 59783c79ddaf4e7aaa2469370832f90c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:37:22.397012 902 raft_consensus.cc:2258] T 3f6e74af6f224208894ddab25cfd6c86 P c79e6cecedd34922a54bc7f8962a48dd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:37:22.397161 903 ts_tablet_manager.cc:1905] T 3f6e74af6f224208894ddab25cfd6c86 P 59783c79ddaf4e7aaa2469370832f90c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:37:22.398908 902 ts_tablet_manager.cc:1905] T 3f6e74af6f224208894ddab25cfd6c86 P c79e6cecedd34922a54bc7f8962a48dd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:37:22.400825 903 ts_tablet_manager.cc:1918] T 3f6e74af6f224208894ddab25cfd6c86 P 59783c79ddaf4e7aaa2469370832f90c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:37:22.400933 903 log.cc:1196] T 3f6e74af6f224208894ddab25cfd6c86 P 59783c79ddaf4e7aaa2469370832f90c: Deleting WAL directory at /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786127579900523-11321-0/minicluster-data/ts-1/wal/wals/3f6e74af6f224208894ddab25cfd6c86
I20260807 18:37:22.401305 902 ts_tablet_manager.cc:1918] T 3f6e74af6f224208894ddab25cfd6c86 P c79e6cecedd34922a54bc7f8962a48dd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:37:22.401327 903 ts_tablet_manager.cc:1939] T 3f6e74af6f224208894ddab25cfd6c86 P 59783c79ddaf4e7aaa2469370832f90c: Deleting consensus metadata
I20260807 18:37:22.401427 902 log.cc:1196] T 3f6e74af6f224208894ddab25cfd6c86 P c79e6cecedd34922a54bc7f8962a48dd: Deleting WAL directory at /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786127579900523-11321-0/minicluster-data/ts-2/wal/wals/3f6e74af6f224208894ddab25cfd6c86
I20260807 18:37:22.401748 902 ts_tablet_manager.cc:1939] T 3f6e74af6f224208894ddab25cfd6c86 P c79e6cecedd34922a54bc7f8962a48dd: Deleting consensus metadata
I20260807 18:37:22.402678 336 catalog_manager.cc:4841] TS 59783c79ddaf4e7aaa2469370832f90c (127.11.14.66:34777): tablet 3f6e74af6f224208894ddab25cfd6c86 (table test-table [id=0d6d2b1a18de4524b54dd8b143f38ac4]) successfully deleted
I20260807 18:37:22.403000 342 catalog_manager.cc:4841] TS c79e6cecedd34922a54bc7f8962a48dd (127.11.14.67:39083): tablet 3f6e74af6f224208894ddab25cfd6c86 (table test-table [id=0d6d2b1a18de4524b54dd8b143f38ac4]) successfully deleted
W20260807 18:37:22.405305 336 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.14.65:38027 (error 108)
I20260807 18:37:22.406100 11321 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkz56XN/build/debug/bin/kudu with pid 560
I20260807 18:37:22.406381 868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:22.406055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.65:38027 (local address 127.0.0.1:47092)
0807 18:37:22.406210 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0807 18:37:22.406286 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.65:38027: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260807 18:37:22.407693 336 catalog_manager.cc:4584] TS 919e53b0c66843af85ad640da2d6bcb2 (127.11.14.65:38027): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 3f6e74af6f224208894ddab25cfd6c86: Network error: Client connection negotiation failed: client connection to 127.11.14.65:38027: connect: Connection refused (error 111)
I20260807 18:37:22.417444 11321 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkz56XN/build/debug/bin/kudu with pid 709
I20260807 18:37:22.424882 11321 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkz56XN/build/debug/bin/kudu with pid 320
2026-08-07T18:37:22Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (1308 ms)
[----------] 6 tests from EncryptionPolicyTest (11223 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786127579900523-11321-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:37:22 dist-test-slave-6k22 krb5kdc[909](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:37:22 dist-test-slave-6k22 krb5kdc[909](info): set up 2 sockets
Aug 07 18:37:22 dist-test-slave-6k22 krb5kdc[909](info): commencing operation
krb5kdc: starting...
W20260807 18:37:23.562484 11321 mini_kdc.cc:121] Time spent starting KDC: real 1.113s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786127579900523-11321-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786127579900523-11321-0/krb5kdc/test-user.keytab.
Aug 07 18:37:23 dist-test-slave-6k22 krb5kdc[909](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127843, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:37:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:37:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786127579900523-11321-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786127579900523-11321-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 18:37:23.690992 11321 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkz56XN/build/debug/bin/kudu
/tmp/dist-test-taskkz56XN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786127579900523-11321-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786127579900523-11321-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786127579900523-11321-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786127579900523-11321-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35575
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786127579900523-11321-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36305
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35575
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786127579900523-11321-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786127579900523-11321-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786127579900523-11321-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:37:23.807938 923 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:37:23.808214 923 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:37:23.808256 923 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:37:23.812297 923 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:37:23.812359 923 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:37:23.812378 923 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:37:23.812472 923 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:37:23.812506 923 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:37:23.817986 923 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36305
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786127579900523-11321-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786127579900523-11321-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35575
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786127579900523-11321-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35575
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786127579900523-11321-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.923
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786127579900523-11321-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:37:23.819154 923 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:37:23.820232 923 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:37:23.828872 932 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:37:23.828826 928 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:37:23.828938 929 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:37:23.828977 923 server_base.cc:1034] running on GCE node
I20260807 18:37:23.829749 923 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:37:23.830901 923 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:37:23.834189 923 hybrid_clock.cc:648] HybridClock initialized: now 1786127843834176 us; error 58 us; skew 500 ppm
Aug 07 18:37:23 dist-test-slave-6k22 krb5kdc[909](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127843, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:37:23.837105 923 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 18:37:23.838058 923 webserver.cc:460] Webserver started at http://127.0.0.1:38225/ using document root <none> and password file <none>
I20260807 18:37:23.838614 923 fs_manager.cc:359] Metadata directory not provided
I20260807 18:37:23.838675 923 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:37:23.838847 923 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:37:23.840991 923 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786127579900523-11321-0/minicluster-data/master-0/data/instance:
uuid: "a2ea2e715d514513a1fe1aceae55b913"
format_stamp: "Formatted at 2026-08-07 18:37:23 on dist-test-slave-6k22"
server_key: "7c3d545bd23a44187397a2a431d1d8a9"
server_key_iv: "1a6359b0ff57a8481c2de77c014b1e73"
server_key_version: "encryptionkey@0"
I20260807 18:37:23.841637 923 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786127579900523-11321-0/minicluster-data/master-0/wal/instance:
uuid: "a2ea2e715d514513a1fe1aceae55b913"
format_stamp: "Formatted at 2026-08-07 18:37:23 on dist-test-slave-6k22"
server_key: "7c3d545bd23a44187397a2a431d1d8a9"
server_key_iv: "1a6359b0ff57a8481c2de77c014b1e73"
server_key_version: "encryptionkey@0"
I20260807 18:37:23.845608 923 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 18:37:23.848130 940 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:37:23.849538 923 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:37:23.849639 923 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786127579900523-11321-0/minicluster-data/master-0/data,/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786127579900523-11321-0/minicluster-data/master-0/wal
uuid: "a2ea2e715d514513a1fe1aceae55b913"
format_stamp: "Formatted at 2026-08-07 18:37:23 on dist-test-slave-6k22"
server_key: "7c3d545bd23a44187397a2a431d1d8a9"
server_key_iv: "1a6359b0ff57a8481c2de77c014b1e73"
server_key_version: "encryptionkey@0"
I20260807 18:37:23.849725 923 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786127579900523-11321-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786127579900523-11321-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786127579900523-11321-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:37:23.870911 923 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:37:23.874017 923 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:37:23.874177 923 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:37:23.881976 923 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35575
I20260807 18:37:23.882004 994 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35575 every 8 connection(s)
I20260807 18:37:23.882992 923 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786127579900523-11321-0/minicluster-data/master-0/data/info.pb
I20260807 18:37:23.886230 995 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:37:23.888388 11321 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkz56XN/build/debug/bin/kudu as pid 923
I20260807 18:37:23.888505 11321 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786127579900523-11321-0/minicluster-data/master-0/wal/instance
I20260807 18:37:23.888828 11321 external_mini_cluster.cc:1428] Setting key 56177e71f8106e3259bd888e1bfbf283
I20260807 18:37:23.893822 995 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a2ea2e715d514513a1fe1aceae55b913: Bootstrap starting.
I20260807 18:37:23.896756 995 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a2ea2e715d514513a1fe1aceae55b913: Neither blocks nor log segments found. Creating new log.
I20260807 18:37:23.897594 995 log.cc:824] T 00000000000000000000000000000000 P a2ea2e715d514513a1fe1aceae55b913: Log is configured to *not* fsync() on all Append() calls
Aug 07 18:37:23 dist-test-slave-6k22 krb5kdc[909](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127843, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260807 18:37:23.899986 995 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a2ea2e715d514513a1fe1aceae55b913: No bootstrap required, opened a new log
I20260807 18:37:23.905105 998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:23.890370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35761 (local address 127.0.0.1:35575)
0807 18:37:23.890830 (+ 460us) server_negotiation.cc:206] Beginning negotiation
0807 18:37:23.890853 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0807 18:37:23.890902 (+ 49us) server_negotiation.cc:407] Connection header received
0807 18:37:23.891820 (+ 918us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:37:23.891849 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:37:23.892325 (+ 476us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:37:23.892742 (+ 417us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:37:23.893426 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:23.894912 (+ 1486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:23.895864 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:23.896555 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:23.896760 (+ 205us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:23.899411 (+ 2651us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:37:23.899419 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:37:23.899441 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:37:23.899475 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:37:23.901632 (+ 2157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:37:23.902145 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:37:23.902152 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:37:23.902158 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:37:23.902317 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:37:23.902673 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:37:23.902679 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:37:23.902682 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:37:23.903270 (+ 588us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:37:23.903511 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0807 18:37:23.903793 (+ 282us) server_negotiation.cc:299] Negotiation successful
0807 18:37:23.904040 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":301,"thread_start_us":114,"threads_started":1}
I20260807 18:37:23.905109 995 raft_consensus.cc:348] T 00000000000000000000000000000000 P a2ea2e715d514513a1fe1aceae55b913 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2ea2e715d514513a1fe1aceae55b913" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35575 } }
I20260807 18:37:23.905431 995 raft_consensus.cc:374] T 00000000000000000000000000000000 P a2ea2e715d514513a1fe1aceae55b913 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:37:23.905494 995 raft_consensus.cc:729] T 00000000000000000000000000000000 P a2ea2e715d514513a1fe1aceae55b913 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2ea2e715d514513a1fe1aceae55b913, State: Initialized, Role: FOLLOWER
I20260807 18:37:23.905918 995 consensus_queue.cc:260] T 00000000000000000000000000000000 P a2ea2e715d514513a1fe1aceae55b913 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2ea2e715d514513a1fe1aceae55b913" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35575 } }
I20260807 18:37:23.906033 995 raft_consensus.cc:388] T 00000000000000000000000000000000 P a2ea2e715d514513a1fe1aceae55b913 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:37:23.906076 995 raft_consensus.cc:482] T 00000000000000000000000000000000 P a2ea2e715d514513a1fe1aceae55b913 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:37:23.906159 995 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a2ea2e715d514513a1fe1aceae55b913 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:37:23.907167 995 raft_consensus.cc:504] T 00000000000000000000000000000000 P a2ea2e715d514513a1fe1aceae55b913 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2ea2e715d514513a1fe1aceae55b913" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35575 } }
I20260807 18:37:23.907603 995 leader_election.cc:302] T 00000000000000000000000000000000 P a2ea2e715d514513a1fe1aceae55b913 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a2ea2e715d514513a1fe1aceae55b913; no voters:
I20260807 18:37:23.907961 995 leader_election.cc:288] T 00000000000000000000000000000000 P a2ea2e715d514513a1fe1aceae55b913 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:37:23.908097 1000 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a2ea2e715d514513a1fe1aceae55b913 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:37:23.908326 1000 raft_consensus.cc:686] T 00000000000000000000000000000000 P a2ea2e715d514513a1fe1aceae55b913 [term 1 LEADER]: Becoming Leader. State: Replica: a2ea2e715d514513a1fe1aceae55b913, State: Running, Role: LEADER
I20260807 18:37:23.908411 11321 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 18:37:23.908655 1000 consensus_queue.cc:237] T 00000000000000000000000000000000 P a2ea2e715d514513a1fe1aceae55b913 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2ea2e715d514513a1fe1aceae55b913" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35575 } }
I20260807 18:37:23.909318 995 sys_catalog.cc:564] T 00000000000000000000000000000000 P a2ea2e715d514513a1fe1aceae55b913 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:37:23.909952 1000 sys_catalog.cc:455] T 00000000000000000000000000000000 P a2ea2e715d514513a1fe1aceae55b913 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a2ea2e715d514513a1fe1aceae55b913" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2ea2e715d514513a1fe1aceae55b913" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35575 } } }
I20260807 18:37:23.910076 1000 sys_catalog.cc:458] T 00000000000000000000000000000000 P a2ea2e715d514513a1fe1aceae55b913 [sys.catalog]: This master's current role is: LEADER
I20260807 18:37:23.910292 1000 sys_catalog.cc:455] T 00000000000000000000000000000000 P a2ea2e715d514513a1fe1aceae55b913 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a2ea2e715d514513a1fe1aceae55b913. Latest consensus state: current_term: 1 leader_uuid: "a2ea2e715d514513a1fe1aceae55b913" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2ea2e715d514513a1fe1aceae55b913" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35575 } } }
I20260807 18:37:23.910382 1000 sys_catalog.cc:458] T 00000000000000000000000000000000 P a2ea2e715d514513a1fe1aceae55b913 [sys.catalog]: This master's current role is: LEADER
W20260807 18:37:23.915315 1022 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a2ea2e715d514513a1fe1aceae55b913: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 18:37:23.915416 1022 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 18:37:23.915527 1010 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:37:23.916633 1010 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:37:23.921022 998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:23.910751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36179 (local address 127.0.0.1:35575)
0807 18:37:23.910866 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0807 18:37:23.910870 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:37:23.911080 (+ 210us) server_negotiation.cc:407] Connection header received
0807 18:37:23.911304 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:37:23.911310 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:37:23.911375 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:37:23.911485 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:37:23.912310 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:23.913054 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:23.914357 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:23.914719 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:23.914858 (+ 139us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:23.916710 (+ 1852us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:37:23.916714 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:37:23.916717 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:37:23.916744 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:37:23.918565 (+ 1821us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:37:23.919220 (+ 655us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:37:23.919225 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:37:23.919228 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:37:23.919372 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:37:23.919818 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:37:23.919821 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:37:23.919823 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:37:23.920052 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:37:23.920207 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0807 18:37:23.920561 (+ 354us) server_negotiation.cc:299] Negotiation successful
0807 18:37:23.920857 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260807 18:37:23.923657 1010 catalog_manager.cc:1347] Generated new cluster ID: 2c35dcd67084488db999dab308dcdd8b
I20260807 18:37:23.923720 1010 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:37:23.972388 1010 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:37:23.973556 1010 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:37:23.986344 1010 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a2ea2e715d514513a1fe1aceae55b913: Generated new TSK 0
I20260807 18:37:23.987212 1010 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:37:23.993907 11321 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkz56XN/build/debug/bin/kudu with pid 923
W20260807 18:37:24.002591 906 connection.cc:535] client connection to 127.0.0.1:35575 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:35575): Connection reset by peer (error 104)
2026-08-07T18:37:23Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1567 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786127579900523-11321-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:37:24 dist-test-slave-6k22 krb5kdc[1030](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:37:24 dist-test-slave-6k22 krb5kdc[1030](info): set up 2 sockets
Aug 07 18:37:24 dist-test-slave-6k22 krb5kdc[1030](info): commencing operation
krb5kdc: starting...
W20260807 18:37:25.138785 11321 mini_kdc.cc:121] Time spent starting KDC: real 1.122s user 0.001s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786127579900523-11321-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786127579900523-11321-0/krb5kdc/test-user.keytab.
Aug 07 18:37:25 dist-test-slave-6k22 krb5kdc[1030](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127845, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:37:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:37:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786127579900523-11321-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786127579900523-11321-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 18:37:25.275337 11321 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkz56XN/build/debug/bin/kudu
/tmp/dist-test-taskkz56XN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786127579900523-11321-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786127579900523-11321-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786127579900523-11321-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786127579900523-11321-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:32959
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786127579900523-11321-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40449
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:32959
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786127579900523-11321-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786127579900523-11321-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786127579900523-11321-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:37:25.396788 1044 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:37:25.397127 1044 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:37:25.397226 1044 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:37:25.401319 1044 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:37:25.401423 1044 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:37:25.401463 1044 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:37:25.401499 1044 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:37:25.401532 1044 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:37:25.406937 1044 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40449
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786127579900523-11321-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786127579900523-11321-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:32959
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786127579900523-11321-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:32959
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786127579900523-11321-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1044
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786127579900523-11321-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:37:25.408103 1044 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:37:25.409166 1044 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:37:25.418020 1052 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:37:25.418030 1050 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:37:25.418046 1049 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:37:25.418421 1044 server_base.cc:1034] running on GCE node
I20260807 18:37:25.419034 1044 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:37:25.420336 1044 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:37:25.421576 1044 hybrid_clock.cc:648] HybridClock initialized: now 1786127845421532 us; error 56 us; skew 500 ppm
Aug 07 18:37:25 dist-test-slave-6k22 krb5kdc[1030](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127845, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:37:25.424786 1044 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 18:37:25.425726 1044 webserver.cc:460] Webserver started at http://127.0.0.1:37171/ using document root <none> and password file <none>
I20260807 18:37:25.426338 1044 fs_manager.cc:359] Metadata directory not provided
I20260807 18:37:25.426434 1044 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:37:25.426682 1044 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:37:25.428822 1044 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786127579900523-11321-0/minicluster-data/master-0/data/instance:
uuid: "8b0c7391825a485aa0095ecd163c6bda"
format_stamp: "Formatted at 2026-08-07 18:37:25 on dist-test-slave-6k22"
server_key: "d54fd4f2bed185a5e61ffeba64c00898"
server_key_iv: "9acb73f7a6456af06dd1924ba2142e81"
server_key_version: "encryptionkey@0"
I20260807 18:37:25.429401 1044 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786127579900523-11321-0/minicluster-data/master-0/wal/instance:
uuid: "8b0c7391825a485aa0095ecd163c6bda"
format_stamp: "Formatted at 2026-08-07 18:37:25 on dist-test-slave-6k22"
server_key: "d54fd4f2bed185a5e61ffeba64c00898"
server_key_iv: "9acb73f7a6456af06dd1924ba2142e81"
server_key_version: "encryptionkey@0"
I20260807 18:37:25.433322 1044 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 18:37:25.435985 1063 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:37:25.437268 1044 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:37:25.437425 1044 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786127579900523-11321-0/minicluster-data/master-0/data,/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786127579900523-11321-0/minicluster-data/master-0/wal
uuid: "8b0c7391825a485aa0095ecd163c6bda"
format_stamp: "Formatted at 2026-08-07 18:37:25 on dist-test-slave-6k22"
server_key: "d54fd4f2bed185a5e61ffeba64c00898"
server_key_iv: "9acb73f7a6456af06dd1924ba2142e81"
server_key_version: "encryptionkey@0"
I20260807 18:37:25.437541 1044 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786127579900523-11321-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786127579900523-11321-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786127579900523-11321-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:37:25.452594 1044 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:37:25.456072 1044 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:37:25.456298 1044 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:37:25.463598 1044 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:32959
I20260807 18:37:25.463635 1121 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:32959 every 8 connection(s)
I20260807 18:37:25.464629 1044 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786127579900523-11321-0/minicluster-data/master-0/data/info.pb
I20260807 18:37:25.467576 1122 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:37:25.473420 1122 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8b0c7391825a485aa0095ecd163c6bda: Bootstrap starting.
I20260807 18:37:25.473492 11321 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkz56XN/build/debug/bin/kudu as pid 1044
I20260807 18:37:25.473634 11321 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786127579900523-11321-0/minicluster-data/master-0/wal/instance
I20260807 18:37:25.473943 11321 external_mini_cluster.cc:1428] Setting key ff65fed894fbaf8fcc35d4904eea22b2
I20260807 18:37:25.476418 1122 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8b0c7391825a485aa0095ecd163c6bda: Neither blocks nor log segments found. Creating new log.
I20260807 18:37:25.477350 1122 log.cc:824] T 00000000000000000000000000000000 P 8b0c7391825a485aa0095ecd163c6bda: Log is configured to *not* fsync() on all Append() calls
I20260807 18:37:25.479846 1122 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8b0c7391825a485aa0095ecd163c6bda: No bootstrap required, opened a new log
I20260807 18:37:25.484433 1122 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8b0c7391825a485aa0095ecd163c6bda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b0c7391825a485aa0095ecd163c6bda" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32959 } }
I20260807 18:37:25.484727 1122 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8b0c7391825a485aa0095ecd163c6bda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:37:25.484788 1126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:25.475480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42467 (local address 127.0.0.1:32959)
0807 18:37:25.476014 (+ 534us) server_negotiation.cc:206] Beginning negotiation
0807 18:37:25.476034 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0807 18:37:25.476078 (+ 44us) server_negotiation.cc:407] Connection header received
0807 18:37:25.476995 (+ 917us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:37:25.477021 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:37:25.477486 (+ 465us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:37:25.477880 (+ 394us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:37:25.478276 (+ 396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:25.479796 (+ 1520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:25.480738 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:25.481419 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:25.481586 (+ 167us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:25.482496 (+ 910us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:37:25.482505 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:37:25.482524 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:37:25.483144 (+ 620us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:37:25.483215 (+ 71us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:37:25.483248 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:37:25.483379 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0807 18:37:25.483630 (+ 251us) server_negotiation.cc:299] Negotiation successful
0807 18:37:25.483787 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":374,"thread_start_us":140,"threads_started":1}
I20260807 18:37:25.484809 1122 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8b0c7391825a485aa0095ecd163c6bda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b0c7391825a485aa0095ecd163c6bda, State: Initialized, Role: FOLLOWER
I20260807 18:37:25.485282 1122 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8b0c7391825a485aa0095ecd163c6bda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b0c7391825a485aa0095ecd163c6bda" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32959 } }
I20260807 18:37:25.485445 1122 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8b0c7391825a485aa0095ecd163c6bda [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:37:25.485536 1122 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8b0c7391825a485aa0095ecd163c6bda [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:37:25.485664 1122 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8b0c7391825a485aa0095ecd163c6bda [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:37:25.486591 1122 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8b0c7391825a485aa0095ecd163c6bda [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b0c7391825a485aa0095ecd163c6bda" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32959 } }
I20260807 18:37:25.486994 1122 leader_election.cc:302] T 00000000000000000000000000000000 P 8b0c7391825a485aa0095ecd163c6bda [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8b0c7391825a485aa0095ecd163c6bda; no voters:
I20260807 18:37:25.487388 1122 leader_election.cc:288] T 00000000000000000000000000000000 P 8b0c7391825a485aa0095ecd163c6bda [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:37:25.487455 1128 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8b0c7391825a485aa0095ecd163c6bda [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:37:25.487701 1128 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8b0c7391825a485aa0095ecd163c6bda [term 1 LEADER]: Becoming Leader. State: Replica: 8b0c7391825a485aa0095ecd163c6bda, State: Running, Role: LEADER
I20260807 18:37:25.488015 1128 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8b0c7391825a485aa0095ecd163c6bda [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b0c7391825a485aa0095ecd163c6bda" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32959 } }
I20260807 18:37:25.488058 11321 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 18:37:25.488768 1122 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8b0c7391825a485aa0095ecd163c6bda [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:37:25.489709 1129 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8b0c7391825a485aa0095ecd163c6bda [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8b0c7391825a485aa0095ecd163c6bda. Latest consensus state: current_term: 1 leader_uuid: "8b0c7391825a485aa0095ecd163c6bda" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b0c7391825a485aa0095ecd163c6bda" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32959 } } }
I20260807 18:37:25.489857 1129 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8b0c7391825a485aa0095ecd163c6bda [sys.catalog]: This master's current role is: LEADER
I20260807 18:37:25.490199 1130 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8b0c7391825a485aa0095ecd163c6bda [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8b0c7391825a485aa0095ecd163c6bda" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b0c7391825a485aa0095ecd163c6bda" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32959 } } }
I20260807 18:37:25.490286 1130 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8b0c7391825a485aa0095ecd163c6bda [sys.catalog]: This master's current role is: LEADER
I20260807 18:37:25.491003 1142 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:37:25.494660 1142 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:37:25.495270 1126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:25.490504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58693 (local address 127.0.0.1:32959)
0807 18:37:25.490767 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0807 18:37:25.490771 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:37:25.491333 (+ 562us) server_negotiation.cc:407] Connection header received
0807 18:37:25.491527 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:37:25.491530 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:37:25.491598 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:37:25.491676 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:37:25.491967 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:25.492702 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:25.493563 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:25.493805 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:25.493918 (+ 113us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:25.494815 (+ 897us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:37:25.494819 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:37:25.494821 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:37:25.494835 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:37:25.494866 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:37:25.494877 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:37:25.494972 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0807 18:37:25.495095 (+ 123us) server_negotiation.cc:299] Negotiation successful
0807 18:37:25.495130 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":182}
I20260807 18:37:25.500875 1142 catalog_manager.cc:1347] Generated new cluster ID: 8c7d7722f1bb4eaf9717ad7df74e72dd
I20260807 18:37:25.500984 1142 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:37:25.526274 1142 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:37:25.527539 1142 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:37:25.534258 1142 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8b0c7391825a485aa0095ecd163c6bda: Generated new TSK 0
I20260807 18:37:25.535132 1142 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:37:25.543395 11321 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkz56XN/build/debug/bin/kudu with pid 1044
W20260807 18:37:25.551543 1027 connection.cc:535] client connection to 127.0.0.1:32959 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:32959): Connection reset by peer (error 104)
2026-08-07T18:37:25Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1549 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786127579900523-11321-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:37:25 dist-test-slave-6k22 krb5kdc[1155](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:37:25 dist-test-slave-6k22 krb5kdc[1155](info): set up 2 sockets
Aug 07 18:37:25 dist-test-slave-6k22 krb5kdc[1155](info): commencing operation
krb5kdc: starting...
W20260807 18:37:26.708632 11321 mini_kdc.cc:121] Time spent starting KDC: real 1.142s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786127579900523-11321-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786127579900523-11321-0/krb5kdc/test-user.keytab.
Aug 07 18:37:26 dist-test-slave-6k22 krb5kdc[1155](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127846, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:37:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:37:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786127579900523-11321-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786127579900523-11321-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 18:37:26.843348 11321 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkz56XN/build/debug/bin/kudu
/tmp/dist-test-taskkz56XN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786127579900523-11321-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786127579900523-11321-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786127579900523-11321-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786127579900523-11321-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43853
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786127579900523-11321-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43159
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43853
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786127579900523-11321-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786127579900523-11321-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786127579900523-11321-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:37:26.962736 1168 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:37:26.963011 1168 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:37:26.963054 1168 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:37:26.967989 1168 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:37:26.968070 1168 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:37:26.968092 1168 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:37:26.968111 1168 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:37:26.968127 1168 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:37:26.974140 1168 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43159
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786127579900523-11321-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786127579900523-11321-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43853
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786127579900523-11321-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43853
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786127579900523-11321-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1168
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786127579900523-11321-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:37:26.975322 1168 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:37:26.976420 1168 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:37:26.986220 1177 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:37:26.986188 1175 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:37:26.986254 1174 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:37:26.986799 1168 server_base.cc:1034] running on GCE node
I20260807 18:37:26.987447 1168 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:37:26.988683 1168 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:37:26.989897 1168 hybrid_clock.cc:648] HybridClock initialized: now 1786127846989880 us; error 29 us; skew 500 ppm
Aug 07 18:37:26 dist-test-slave-6k22 krb5kdc[1155](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127846, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:37:27.001832 1168 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 18:37:27.003000 1168 webserver.cc:460] Webserver started at http://127.0.0.1:39403/ using document root <none> and password file <none>
I20260807 18:37:27.003794 1168 fs_manager.cc:359] Metadata directory not provided
I20260807 18:37:27.003901 1168 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:37:27.004199 1168 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:37:27.006531 1168 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786127579900523-11321-0/minicluster-data/master-0/data/instance:
uuid: "b6d354baf41b48d989ff6dfd3d44a2ca"
format_stamp: "Formatted at 2026-08-07 18:37:27 on dist-test-slave-6k22"
server_key: "20dcd21a1a7cbd478e65e4938b52a823"
server_key_iv: "b09a1ab523c186f69b4e7009ba804c6d"
server_key_version: "encryptionkey@0"
I20260807 18:37:27.007252 1168 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786127579900523-11321-0/minicluster-data/master-0/wal/instance:
uuid: "b6d354baf41b48d989ff6dfd3d44a2ca"
format_stamp: "Formatted at 2026-08-07 18:37:27 on dist-test-slave-6k22"
server_key: "20dcd21a1a7cbd478e65e4938b52a823"
server_key_iv: "b09a1ab523c186f69b4e7009ba804c6d"
server_key_version: "encryptionkey@0"
I20260807 18:37:27.011512 1168 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.003s
I20260807 18:37:27.014353 1185 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:37:27.015784 1168 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:37:27.015944 1168 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786127579900523-11321-0/minicluster-data/master-0/data,/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786127579900523-11321-0/minicluster-data/master-0/wal
uuid: "b6d354baf41b48d989ff6dfd3d44a2ca"
format_stamp: "Formatted at 2026-08-07 18:37:27 on dist-test-slave-6k22"
server_key: "20dcd21a1a7cbd478e65e4938b52a823"
server_key_iv: "b09a1ab523c186f69b4e7009ba804c6d"
server_key_version: "encryptionkey@0"
I20260807 18:37:27.016069 1168 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786127579900523-11321-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786127579900523-11321-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786127579900523-11321-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:37:27.021333 1168 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:37:27.024547 1168 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:37:27.024899 1168 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:37:27.032415 1168 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43853
I20260807 18:37:27.032445 1244 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43853 every 8 connection(s)
I20260807 18:37:27.033380 1168 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786127579900523-11321-0/minicluster-data/master-0/data/info.pb
I20260807 18:37:27.036224 1245 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:37:27.041074 11321 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkz56XN/build/debug/bin/kudu as pid 1168
I20260807 18:37:27.041236 11321 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786127579900523-11321-0/minicluster-data/master-0/wal/instance
I20260807 18:37:27.041524 11321 external_mini_cluster.cc:1428] Setting key 0af6f8303056976da44fceb9a1788209
I20260807 18:37:27.041831 1245 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b6d354baf41b48d989ff6dfd3d44a2ca: Bootstrap starting.
I20260807 18:37:27.044744 1245 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b6d354baf41b48d989ff6dfd3d44a2ca: Neither blocks nor log segments found. Creating new log.
I20260807 18:37:27.045687 1245 log.cc:824] T 00000000000000000000000000000000 P b6d354baf41b48d989ff6dfd3d44a2ca: Log is configured to *not* fsync() on all Append() calls
I20260807 18:37:27.047523 1248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:27.043072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47279 (local address 127.0.0.1:43853)
0807 18:37:27.043522 (+ 450us) server_negotiation.cc:206] Beginning negotiation
0807 18:37:27.043542 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0807 18:37:27.043605 (+ 63us) server_negotiation.cc:407] Connection header received
0807 18:37:27.044423 (+ 818us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:37:27.044447 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:37:27.044853 (+ 406us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:37:27.045237 (+ 384us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:37:27.046098 (+ 861us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:37:27.046106 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:37:27.046127 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:37:27.046138 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:37:27.046190 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:37:27.046209 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:37:27.046297 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0807 18:37:27.046454 (+ 157us) server_negotiation.cc:299] Negotiation successful
0807 18:37:27.046627 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":287,"thread_start_us":119,"threads_started":1}
I20260807 18:37:27.047981 1245 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b6d354baf41b48d989ff6dfd3d44a2ca: No bootstrap required, opened a new log
I20260807 18:37:27.050629 11321 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 18:37:27.052362 1245 raft_consensus.cc:348] T 00000000000000000000000000000000 P b6d354baf41b48d989ff6dfd3d44a2ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6d354baf41b48d989ff6dfd3d44a2ca" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43853 } }
I20260807 18:37:27.052632 1245 raft_consensus.cc:374] T 00000000000000000000000000000000 P b6d354baf41b48d989ff6dfd3d44a2ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:37:27.052723 1245 raft_consensus.cc:729] T 00000000000000000000000000000000 P b6d354baf41b48d989ff6dfd3d44a2ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b6d354baf41b48d989ff6dfd3d44a2ca, State: Initialized, Role: FOLLOWER
I20260807 18:37:27.053244 1245 consensus_queue.cc:260] T 00000000000000000000000000000000 P b6d354baf41b48d989ff6dfd3d44a2ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6d354baf41b48d989ff6dfd3d44a2ca" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43853 } }
I20260807 18:37:27.053434 1245 raft_consensus.cc:388] T 00000000000000000000000000000000 P b6d354baf41b48d989ff6dfd3d44a2ca [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:37:27.053519 1245 raft_consensus.cc:482] T 00000000000000000000000000000000 P b6d354baf41b48d989ff6dfd3d44a2ca [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:37:27.053637 1245 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b6d354baf41b48d989ff6dfd3d44a2ca [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:37:27.054920 1245 raft_consensus.cc:504] T 00000000000000000000000000000000 P b6d354baf41b48d989ff6dfd3d44a2ca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6d354baf41b48d989ff6dfd3d44a2ca" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43853 } }
I20260807 18:37:27.055007 1248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:27.052895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40303 (local address 127.0.0.1:43853)
0807 18:37:27.053025 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0807 18:37:27.053031 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:37:27.053328 (+ 297us) server_negotiation.cc:407] Connection header received
0807 18:37:27.053409 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:37:27.053412 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:37:27.053448 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:37:27.053533 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:37:27.054465 (+ 932us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:37:27.054468 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:37:27.054471 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:37:27.054483 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:37:27.054510 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:37:27.054525 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:37:27.054621 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0807 18:37:27.054803 (+ 182us) server_negotiation.cc:299] Negotiation successful
0807 18:37:27.054846 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260807 18:37:27.055315 1245 leader_election.cc:302] T 00000000000000000000000000000000 P b6d354baf41b48d989ff6dfd3d44a2ca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b6d354baf41b48d989ff6dfd3d44a2ca; no voters:
I20260807 18:37:27.055698 1245 leader_election.cc:288] T 00000000000000000000000000000000 P b6d354baf41b48d989ff6dfd3d44a2ca [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:37:27.055760 1255 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b6d354baf41b48d989ff6dfd3d44a2ca [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:37:27.056092 1255 raft_consensus.cc:686] T 00000000000000000000000000000000 P b6d354baf41b48d989ff6dfd3d44a2ca [term 1 LEADER]: Becoming Leader. State: Replica: b6d354baf41b48d989ff6dfd3d44a2ca, State: Running, Role: LEADER
I20260807 18:37:27.056397 1255 consensus_queue.cc:237] T 00000000000000000000000000000000 P b6d354baf41b48d989ff6dfd3d44a2ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6d354baf41b48d989ff6dfd3d44a2ca" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43853 } }
I20260807 18:37:27.057019 1245 sys_catalog.cc:564] T 00000000000000000000000000000000 P b6d354baf41b48d989ff6dfd3d44a2ca [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:37:27.057605 1255 sys_catalog.cc:455] T 00000000000000000000000000000000 P b6d354baf41b48d989ff6dfd3d44a2ca [sys.catalog]: SysCatalogTable state changed. Reason: New leader b6d354baf41b48d989ff6dfd3d44a2ca. Latest consensus state: current_term: 1 leader_uuid: "b6d354baf41b48d989ff6dfd3d44a2ca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6d354baf41b48d989ff6dfd3d44a2ca" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43853 } } }
I20260807 18:37:27.057723 1255 sys_catalog.cc:458] T 00000000000000000000000000000000 P b6d354baf41b48d989ff6dfd3d44a2ca [sys.catalog]: This master's current role is: LEADER
I20260807 18:37:27.058418 1257 sys_catalog.cc:455] T 00000000000000000000000000000000 P b6d354baf41b48d989ff6dfd3d44a2ca [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b6d354baf41b48d989ff6dfd3d44a2ca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6d354baf41b48d989ff6dfd3d44a2ca" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43853 } } }
I20260807 18:37:27.058518 1257 sys_catalog.cc:458] T 00000000000000000000000000000000 P b6d354baf41b48d989ff6dfd3d44a2ca [sys.catalog]: This master's current role is: LEADER
W20260807 18:37:27.061971 1272 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b6d354baf41b48d989ff6dfd3d44a2ca: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 18:37:27.062073 1272 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 18:37:27.062158 1262 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:37:27.062897 1262 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:37:27.068917 1262 catalog_manager.cc:1347] Generated new cluster ID: 42462966abaa439a878edf759fea25d6
I20260807 18:37:27.068984 1262 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:37:27.090262 1262 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:37:27.090595 1262 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:37:27.096395 1262 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b6d354baf41b48d989ff6dfd3d44a2ca: Generated new TSK 0
I20260807 18:37:27.097141 1262 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:37:27.107764 11321 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkz56XN/build/debug/bin/kudu with pid 1168
2026-08-07T18:37:27Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1565 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786127579900523-11321-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:37:27 dist-test-slave-6k22 krb5kdc[1279](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:37:27 dist-test-slave-6k22 krb5kdc[1279](info): set up 2 sockets
Aug 07 18:37:27 dist-test-slave-6k22 krb5kdc[1279](info): commencing operation
krb5kdc: starting...
W20260807 18:37:28.297583 11321 mini_kdc.cc:121] Time spent starting KDC: real 1.166s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786127579900523-11321-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786127579900523-11321-0/krb5kdc/test-user.keytab.
Aug 07 18:37:28 dist-test-slave-6k22 krb5kdc[1279](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127848, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:37:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:37:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786127579900523-11321-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786127579900523-11321-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 18:37:28.430811 11321 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkz56XN/build/debug/bin/kudu
/tmp/dist-test-taskkz56XN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786127579900523-11321-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786127579900523-11321-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786127579900523-11321-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786127579900523-11321-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42979
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786127579900523-11321-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36785
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42979
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786127579900523-11321-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786127579900523-11321-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786127579900523-11321-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:37:28.548903 1292 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:37:28.549235 1292 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:37:28.549317 1292 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:37:28.553369 1292 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:37:28.553491 1292 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:37:28.553521 1292 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:37:28.553547 1292 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:37:28.553575 1292 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:37:28.559156 1292 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36785
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786127579900523-11321-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786127579900523-11321-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42979
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786127579900523-11321-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42979
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786127579900523-11321-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1292
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786127579900523-11321-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:37:28.560271 1292 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:37:28.561290 1292 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:37:28.570206 1298 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:37:28.570248 1299 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:37:28.570205 1301 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:37:28.570477 1292 server_base.cc:1034] running on GCE node
I20260807 18:37:28.571058 1292 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:37:28.572294 1292 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:37:28.573467 1292 hybrid_clock.cc:648] HybridClock initialized: now 1786127848573453 us; error 24 us; skew 500 ppm
Aug 07 18:37:28 dist-test-slave-6k22 krb5kdc[1279](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127848, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:37:28.576345 1292 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 18:37:28.577255 1292 webserver.cc:460] Webserver started at http://127.0.0.1:44729/ using document root <none> and password file <none>
I20260807 18:37:28.577903 1292 fs_manager.cc:359] Metadata directory not provided
I20260807 18:37:28.577999 1292 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:37:28.578244 1292 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:37:28.580454 1292 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786127579900523-11321-0/minicluster-data/master-0/data/instance:
uuid: "ffbe4ab7420d43c6b967968708610075"
format_stamp: "Formatted at 2026-08-07 18:37:28 on dist-test-slave-6k22"
server_key: "95f9b0dd03bb985ca5b28a434a757821"
server_key_iv: "4f5f7a9d3da1c7042d5d831cf0c135f0"
server_key_version: "encryptionkey@0"
I20260807 18:37:28.581036 1292 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786127579900523-11321-0/minicluster-data/master-0/wal/instance:
uuid: "ffbe4ab7420d43c6b967968708610075"
format_stamp: "Formatted at 2026-08-07 18:37:28 on dist-test-slave-6k22"
server_key: "95f9b0dd03bb985ca5b28a434a757821"
server_key_iv: "4f5f7a9d3da1c7042d5d831cf0c135f0"
server_key_version: "encryptionkey@0"
I20260807 18:37:28.584919 1292 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 18:37:28.587683 1309 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:37:28.588951 1292 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.001s
I20260807 18:37:28.589120 1292 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786127579900523-11321-0/minicluster-data/master-0/data,/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786127579900523-11321-0/minicluster-data/master-0/wal
uuid: "ffbe4ab7420d43c6b967968708610075"
format_stamp: "Formatted at 2026-08-07 18:37:28 on dist-test-slave-6k22"
server_key: "95f9b0dd03bb985ca5b28a434a757821"
server_key_iv: "4f5f7a9d3da1c7042d5d831cf0c135f0"
server_key_version: "encryptionkey@0"
I20260807 18:37:28.589247 1292 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786127579900523-11321-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786127579900523-11321-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786127579900523-11321-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:37:28.605370 1292 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:37:28.608669 1292 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:37:28.608896 1292 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:37:28.616757 1292 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42979
I20260807 18:37:28.616802 1370 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42979 every 8 connection(s)
I20260807 18:37:28.617835 1292 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786127579900523-11321-0/minicluster-data/master-0/data/info.pb
I20260807 18:37:28.618220 11321 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkz56XN/build/debug/bin/kudu as pid 1292
I20260807 18:37:28.618376 11321 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786127579900523-11321-0/minicluster-data/master-0/wal/instance
I20260807 18:37:28.618712 11321 external_mini_cluster.cc:1428] Setting key bfd39af72991b2768f98a069605f520b
I20260807 18:37:28.621352 1371 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:37:28.628237 1371 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ffbe4ab7420d43c6b967968708610075: Bootstrap starting.
Aug 07 18:37:28 dist-test-slave-6k22 krb5kdc[1279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127848, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260807 18:37:28.630604 1371 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ffbe4ab7420d43c6b967968708610075: Neither blocks nor log segments found. Creating new log.
I20260807 18:37:28.631398 1371 log.cc:824] T 00000000000000000000000000000000 P ffbe4ab7420d43c6b967968708610075: Log is configured to *not* fsync() on all Append() calls
I20260807 18:37:28.633431 1371 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ffbe4ab7420d43c6b967968708610075: No bootstrap required, opened a new log
I20260807 18:37:28.633786 1374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:28.620294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36121 (local address 127.0.0.1:42979)
0807 18:37:28.620738 (+ 444us) server_negotiation.cc:206] Beginning negotiation
0807 18:37:28.620754 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 18:37:28.620792 (+ 38us) server_negotiation.cc:407] Connection header received
0807 18:37:28.621443 (+ 651us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:37:28.621491 (+ 48us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:37:28.621860 (+ 369us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:37:28.622145 (+ 285us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:37:28.623096 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:28.624190 (+ 1094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:28.625417 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:28.625938 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:28.626076 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:28.628993 (+ 2917us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:37:28.628998 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:37:28.629011 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:37:28.629037 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:37:28.630815 (+ 1778us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:37:28.631292 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:37:28.631298 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:37:28.631303 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:37:28.631416 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:37:28.631739 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:37:28.631743 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:37:28.631744 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:37:28.632197 (+ 453us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:37:28.632420 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0807 18:37:28.632736 (+ 316us) server_negotiation.cc:299] Negotiation successful
0807 18:37:28.633020 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":303,"thread_start_us":137,"threads_started":1}
I20260807 18:37:28.636600 11321 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 18:37:28.637281 1371 raft_consensus.cc:348] T 00000000000000000000000000000000 P ffbe4ab7420d43c6b967968708610075 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffbe4ab7420d43c6b967968708610075" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42979 } }
I20260807 18:37:28.637583 1371 raft_consensus.cc:374] T 00000000000000000000000000000000 P ffbe4ab7420d43c6b967968708610075 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:37:28.637650 1371 raft_consensus.cc:729] T 00000000000000000000000000000000 P ffbe4ab7420d43c6b967968708610075 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ffbe4ab7420d43c6b967968708610075, State: Initialized, Role: FOLLOWER
I20260807 18:37:28.638167 1371 consensus_queue.cc:260] T 00000000000000000000000000000000 P ffbe4ab7420d43c6b967968708610075 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffbe4ab7420d43c6b967968708610075" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42979 } }
I20260807 18:37:28.638360 1371 raft_consensus.cc:388] T 00000000000000000000000000000000 P ffbe4ab7420d43c6b967968708610075 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:37:28.638438 1371 raft_consensus.cc:482] T 00000000000000000000000000000000 P ffbe4ab7420d43c6b967968708610075 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:37:28.638541 1371 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ffbe4ab7420d43c6b967968708610075 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:37:28.639752 1371 raft_consensus.cc:504] T 00000000000000000000000000000000 P ffbe4ab7420d43c6b967968708610075 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffbe4ab7420d43c6b967968708610075" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42979 } }
I20260807 18:37:28.640164 1371 leader_election.cc:302] T 00000000000000000000000000000000 P ffbe4ab7420d43c6b967968708610075 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ffbe4ab7420d43c6b967968708610075; no voters:
I20260807 18:37:28.640534 1371 leader_election.cc:288] T 00000000000000000000000000000000 P ffbe4ab7420d43c6b967968708610075 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:37:28.640631 1382 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ffbe4ab7420d43c6b967968708610075 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:37:28.640892 1382 raft_consensus.cc:686] T 00000000000000000000000000000000 P ffbe4ab7420d43c6b967968708610075 [term 1 LEADER]: Becoming Leader. State: Replica: ffbe4ab7420d43c6b967968708610075, State: Running, Role: LEADER
I20260807 18:37:28.641266 1382 consensus_queue.cc:237] T 00000000000000000000000000000000 P ffbe4ab7420d43c6b967968708610075 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffbe4ab7420d43c6b967968708610075" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42979 } }
I20260807 18:37:28.641810 1371 sys_catalog.cc:564] T 00000000000000000000000000000000 P ffbe4ab7420d43c6b967968708610075 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:37:28.643621 1384 sys_catalog.cc:455] T 00000000000000000000000000000000 P ffbe4ab7420d43c6b967968708610075 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ffbe4ab7420d43c6b967968708610075. Latest consensus state: current_term: 1 leader_uuid: "ffbe4ab7420d43c6b967968708610075" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffbe4ab7420d43c6b967968708610075" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42979 } } }
I20260807 18:37:28.643682 1383 sys_catalog.cc:455] T 00000000000000000000000000000000 P ffbe4ab7420d43c6b967968708610075 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ffbe4ab7420d43c6b967968708610075" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffbe4ab7420d43c6b967968708610075" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42979 } } }
I20260807 18:37:28.643843 1383 sys_catalog.cc:458] T 00000000000000000000000000000000 P ffbe4ab7420d43c6b967968708610075 [sys.catalog]: This master's current role is: LEADER
I20260807 18:37:28.643836 1384 sys_catalog.cc:458] T 00000000000000000000000000000000 P ffbe4ab7420d43c6b967968708610075 [sys.catalog]: This master's current role is: LEADER
W20260807 18:37:28.647619 1398 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ffbe4ab7420d43c6b967968708610075: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 18:37:28.647750 1398 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 18:37:28.647889 1399 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:37:28.648072 1374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:28.638936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45407 (local address 127.0.0.1:42979)
0807 18:37:28.639105 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0807 18:37:28.639109 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:37:28.639382 (+ 273us) server_negotiation.cc:407] Connection header received
0807 18:37:28.639499 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:37:28.639501 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:37:28.639541 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:37:28.639648 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:37:28.640497 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:28.641264 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:28.642405 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:28.642805 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:28.642930 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:28.644606 (+ 1676us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:37:28.644611 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:37:28.644614 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:37:28.644640 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:37:28.646304 (+ 1664us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:37:28.646750 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:37:28.646754 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:37:28.646757 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:37:28.646873 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:37:28.647172 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:37:28.647178 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:37:28.647180 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:37:28.647361 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:37:28.647512 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0807 18:37:28.647794 (+ 282us) server_negotiation.cc:299] Negotiation successful
0807 18:37:28.647911 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260807 18:37:28.649098 1399 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:37:28.655771 1399 catalog_manager.cc:1347] Generated new cluster ID: b71845fd0021459488c81ba19ef8cfd4
I20260807 18:37:28.655846 1399 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:37:28.669274 1399 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:37:28.670224 1399 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:37:28.676030 1399 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ffbe4ab7420d43c6b967968708610075: Generated new TSK 0
I20260807 18:37:28.676714 1399 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:37:28.687031 11321 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkz56XN/build/debug/bin/kudu with pid 1292
2026-08-07T18:37:28Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1580 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786127579900523-11321-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:37:28 dist-test-slave-6k22 krb5kdc[1407](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:37:28 dist-test-slave-6k22 krb5kdc[1407](info): set up 2 sockets
Aug 07 18:37:28 dist-test-slave-6k22 krb5kdc[1407](info): commencing operation
krb5kdc: starting...
W20260807 18:37:29.876996 11321 mini_kdc.cc:121] Time spent starting KDC: real 1.165s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786127579900523-11321-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786127579900523-11321-0/krb5kdc/test-user.keytab.
Aug 07 18:37:29 dist-test-slave-6k22 krb5kdc[1407](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127849, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:37:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:37:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786127579900523-11321-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786127579900523-11321-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 18:37:30.013334 11321 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkz56XN/build/debug/bin/kudu
/tmp/dist-test-taskkz56XN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786127579900523-11321-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786127579900523-11321-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786127579900523-11321-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786127579900523-11321-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40881
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786127579900523-11321-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40849
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40881
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786127579900523-11321-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786127579900523-11321-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786127579900523-11321-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:37:30.127751 1422 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:37:30.128073 1422 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:37:30.128155 1422 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:37:30.132258 1422 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:37:30.132347 1422 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:37:30.132386 1422 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:37:30.132447 1422 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:37:30.132489 1422 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:37:30.137799 1422 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40849
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786127579900523-11321-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786127579900523-11321-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40881
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786127579900523-11321-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40881
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786127579900523-11321-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1422
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786127579900523-11321-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:37:30.138936 1422 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:37:30.140101 1422 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:37:30.148947 1430 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:37:30.148989 1429 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:37:30.149045 1432 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:37:30.149173 1422 server_base.cc:1034] running on GCE node
I20260807 18:37:30.149894 1422 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:37:30.151068 1422 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:37:30.152267 1422 hybrid_clock.cc:648] HybridClock initialized: now 1786127850152230 us; error 46 us; skew 500 ppm
Aug 07 18:37:30 dist-test-slave-6k22 krb5kdc[1407](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127850, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:37:30.155004 1422 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 18:37:30.155982 1422 webserver.cc:460] Webserver started at http://127.0.0.1:34593/ using document root <none> and password file <none>
I20260807 18:37:30.156561 1422 fs_manager.cc:359] Metadata directory not provided
I20260807 18:37:30.156664 1422 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:37:30.156901 1422 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:37:30.159040 1422 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786127579900523-11321-0/minicluster-data/master-0/data/instance:
uuid: "7e2a965ef25143348cefc2acc90b9160"
format_stamp: "Formatted at 2026-08-07 18:37:30 on dist-test-slave-6k22"
server_key: "9a2cf6cb40a764c77fa55a530e36dde8"
server_key_iv: "c5074ff91cf87c5adcbd7bb49e71b95c"
server_key_version: "encryptionkey@0"
I20260807 18:37:30.159701 1422 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786127579900523-11321-0/minicluster-data/master-0/wal/instance:
uuid: "7e2a965ef25143348cefc2acc90b9160"
format_stamp: "Formatted at 2026-08-07 18:37:30 on dist-test-slave-6k22"
server_key: "9a2cf6cb40a764c77fa55a530e36dde8"
server_key_iv: "c5074ff91cf87c5adcbd7bb49e71b95c"
server_key_version: "encryptionkey@0"
I20260807 18:37:30.163622 1422 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260807 18:37:30.166114 1440 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:37:30.167366 1422 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 18:37:30.167498 1422 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786127579900523-11321-0/minicluster-data/master-0/data,/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786127579900523-11321-0/minicluster-data/master-0/wal
uuid: "7e2a965ef25143348cefc2acc90b9160"
format_stamp: "Formatted at 2026-08-07 18:37:30 on dist-test-slave-6k22"
server_key: "9a2cf6cb40a764c77fa55a530e36dde8"
server_key_iv: "c5074ff91cf87c5adcbd7bb49e71b95c"
server_key_version: "encryptionkey@0"
I20260807 18:37:30.167632 1422 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786127579900523-11321-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786127579900523-11321-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786127579900523-11321-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:37:30.197238 1422 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:37:30.200421 1422 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:37:30.200631 1422 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:37:30.208866 1422 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40881
I20260807 18:37:30.209198 1499 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40881 every 8 connection(s)
I20260807 18:37:30.209896 1422 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786127579900523-11321-0/minicluster-data/master-0/data/info.pb
I20260807 18:37:30.210966 11321 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkz56XN/build/debug/bin/kudu as pid 1422
I20260807 18:37:30.211143 11321 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786127579900523-11321-0/minicluster-data/master-0/wal/instance
I20260807 18:37:30.211498 11321 external_mini_cluster.cc:1428] Setting key b006dce16a8d4eed558f7079241cf7c2
I20260807 18:37:30.213784 1501 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:37:30.220346 1501 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7e2a965ef25143348cefc2acc90b9160: Bootstrap starting.
I20260807 18:37:30.220997 1505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:30.213034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43935 (local address 127.0.0.1:40881)
0807 18:37:30.213524 (+ 490us) server_negotiation.cc:206] Beginning negotiation
0807 18:37:30.213539 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 18:37:30.213577 (+ 38us) server_negotiation.cc:407] Connection header received
0807 18:37:30.214320 (+ 743us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:37:30.214340 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:37:30.214698 (+ 358us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:37:30.215043 (+ 345us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:37:30.215490 (+ 447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:30.216569 (+ 1079us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:30.217475 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:30.217947 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:30.218123 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:30.219066 (+ 943us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:37:30.219071 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:37:30.219084 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:37:30.219484 (+ 400us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:37:30.219538 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:37:30.219625 (+ 87us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:37:30.219737 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0807 18:37:30.219893 (+ 156us) server_negotiation.cc:299] Negotiation successful
0807 18:37:30.220147 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":345,"thread_start_us":163,"threads_started":1}
I20260807 18:37:30.222920 1501 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7e2a965ef25143348cefc2acc90b9160: Neither blocks nor log segments found. Creating new log.
I20260807 18:37:30.223912 11321 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 18:37:30.223953 1501 log.cc:824] T 00000000000000000000000000000000 P 7e2a965ef25143348cefc2acc90b9160: Log is configured to *not* fsync() on all Append() calls
I20260807 18:37:30.226259 1501 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7e2a965ef25143348cefc2acc90b9160: No bootstrap required, opened a new log
I20260807 18:37:30.230716 1505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:30.226058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34951 (local address 127.0.0.1:40881)
0807 18:37:30.226173 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0807 18:37:30.226177 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:37:30.226377 (+ 200us) server_negotiation.cc:407] Connection header received
0807 18:37:30.226522 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:37:30.226524 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:37:30.226563 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:37:30.226638 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:37:30.226966 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:30.227456 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:30.228732 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:30.229150 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:30.229287 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:30.230288 (+ 1001us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:37:30.230291 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:37:30.230293 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:37:30.230304 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:37:30.230328 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:37:30.230337 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:37:30.230399 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0807 18:37:30.230543 (+ 144us) server_negotiation.cc:299] Negotiation successful
0807 18:37:30.230586 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260807 18:37:30.231493 1501 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7e2a965ef25143348cefc2acc90b9160 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e2a965ef25143348cefc2acc90b9160" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40881 } }
I20260807 18:37:30.231853 1501 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7e2a965ef25143348cefc2acc90b9160 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:37:30.231927 1501 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7e2a965ef25143348cefc2acc90b9160 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e2a965ef25143348cefc2acc90b9160, State: Initialized, Role: FOLLOWER
I20260807 18:37:30.232383 1501 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7e2a965ef25143348cefc2acc90b9160 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e2a965ef25143348cefc2acc90b9160" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40881 } }
I20260807 18:37:30.232548 1501 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7e2a965ef25143348cefc2acc90b9160 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:37:30.232648 1501 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7e2a965ef25143348cefc2acc90b9160 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:37:30.232761 1501 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7e2a965ef25143348cefc2acc90b9160 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:37:30.233752 1501 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7e2a965ef25143348cefc2acc90b9160 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e2a965ef25143348cefc2acc90b9160" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40881 } }
I20260807 18:37:30.234136 1501 leader_election.cc:302] T 00000000000000000000000000000000 P 7e2a965ef25143348cefc2acc90b9160 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7e2a965ef25143348cefc2acc90b9160; no voters:
I20260807 18:37:30.234490 1501 leader_election.cc:288] T 00000000000000000000000000000000 P 7e2a965ef25143348cefc2acc90b9160 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:37:30.234580 1512 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7e2a965ef25143348cefc2acc90b9160 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:37:30.234805 1512 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7e2a965ef25143348cefc2acc90b9160 [term 1 LEADER]: Becoming Leader. State: Replica: 7e2a965ef25143348cefc2acc90b9160, State: Running, Role: LEADER
I20260807 18:37:30.235409 1512 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7e2a965ef25143348cefc2acc90b9160 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e2a965ef25143348cefc2acc90b9160" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40881 } }
I20260807 18:37:30.235975 1501 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7e2a965ef25143348cefc2acc90b9160 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:37:30.237331 1514 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e2a965ef25143348cefc2acc90b9160 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7e2a965ef25143348cefc2acc90b9160" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e2a965ef25143348cefc2acc90b9160" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40881 } } }
I20260807 18:37:30.237541 1514 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e2a965ef25143348cefc2acc90b9160 [sys.catalog]: This master's current role is: LEADER
I20260807 18:37:30.237637 1513 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e2a965ef25143348cefc2acc90b9160 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7e2a965ef25143348cefc2acc90b9160. Latest consensus state: current_term: 1 leader_uuid: "7e2a965ef25143348cefc2acc90b9160" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e2a965ef25143348cefc2acc90b9160" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40881 } } }
I20260807 18:37:30.237735 1513 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e2a965ef25143348cefc2acc90b9160 [sys.catalog]: This master's current role is: LEADER
I20260807 18:37:30.238165 1525 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:37:30.241727 1525 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:37:30.247596 1525 catalog_manager.cc:1347] Generated new cluster ID: 1650cae5a0254074b3d0dcbe7f49f0c4
I20260807 18:37:30.247666 1525 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:37:30.258468 1525 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:37:30.259749 1525 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:37:30.266412 1525 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7e2a965ef25143348cefc2acc90b9160: Generated new TSK 0
I20260807 18:37:30.267186 1525 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:37:30.280771 11321 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkz56XN/build/debug/bin/kudu with pid 1422
2026-08-07T18:37:30Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1592 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786127579900523-11321-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:37:30 dist-test-slave-6k22 krb5kdc[1536](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:37:30 dist-test-slave-6k22 krb5kdc[1536](info): set up 2 sockets
Aug 07 18:37:30 dist-test-slave-6k22 krb5kdc[1536](info): commencing operation
krb5kdc: starting...
W20260807 18:37:31.429140 11321 mini_kdc.cc:121] Time spent starting KDC: real 1.124s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786127579900523-11321-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786127579900523-11321-0/krb5kdc/test-user.keytab.
Aug 07 18:37:31 dist-test-slave-6k22 krb5kdc[1536](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127851, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:37:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:37:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786127579900523-11321-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786127579900523-11321-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 18:37:31.558931 11321 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkz56XN/build/debug/bin/kudu
/tmp/dist-test-taskkz56XN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786127579900523-11321-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786127579900523-11321-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786127579900523-11321-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786127579900523-11321-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40745
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786127579900523-11321-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43675
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40745
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786127579900523-11321-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786127579900523-11321-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786127579900523-11321-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:37:31.673655 1549 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:37:31.673978 1549 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:37:31.674062 1549 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:37:31.678125 1549 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:37:31.678220 1549 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:37:31.678253 1549 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:37:31.678282 1549 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:37:31.678325 1549 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:37:31.683605 1549 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43675
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786127579900523-11321-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786127579900523-11321-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40745
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786127579900523-11321-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40745
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786127579900523-11321-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1549
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786127579900523-11321-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:37:31.684851 1549 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:37:31.686004 1549 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:37:31.694849 1554 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:37:31.694864 1558 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:37:31.694862 1555 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:37:31.695194 1549 server_base.cc:1034] running on GCE node
I20260807 18:37:31.695874 1549 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:37:31.697077 1549 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:37:31.698280 1549 hybrid_clock.cc:648] HybridClock initialized: now 1786127851698254 us; error 25 us; skew 500 ppm
Aug 07 18:37:31 dist-test-slave-6k22 krb5kdc[1536](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127851, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:37:31.701223 1549 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 18:37:31.702131 1549 webserver.cc:460] Webserver started at http://127.0.0.1:42291/ using document root <none> and password file <none>
I20260807 18:37:31.702673 1549 fs_manager.cc:359] Metadata directory not provided
I20260807 18:37:31.702731 1549 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:37:31.702913 1549 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:37:31.705248 1549 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786127579900523-11321-0/minicluster-data/master-0/data/instance:
uuid: "9420fe1ee3094e648e787d7ad1f93ece"
format_stamp: "Formatted at 2026-08-07 18:37:31 on dist-test-slave-6k22"
server_key: "e9b74cf80e30ab463028b9aa6f4b0b71"
server_key_iv: "98a67577ac65292bc2f02f92ae452b75"
server_key_version: "encryptionkey@0"
I20260807 18:37:31.705910 1549 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786127579900523-11321-0/minicluster-data/master-0/wal/instance:
uuid: "9420fe1ee3094e648e787d7ad1f93ece"
format_stamp: "Formatted at 2026-08-07 18:37:31 on dist-test-slave-6k22"
server_key: "e9b74cf80e30ab463028b9aa6f4b0b71"
server_key_iv: "98a67577ac65292bc2f02f92ae452b75"
server_key_version: "encryptionkey@0"
I20260807 18:37:31.709812 1549 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 18:37:31.712366 1565 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:37:31.713588 1549 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 18:37:31.713738 1549 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786127579900523-11321-0/minicluster-data/master-0/data,/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786127579900523-11321-0/minicluster-data/master-0/wal
uuid: "9420fe1ee3094e648e787d7ad1f93ece"
format_stamp: "Formatted at 2026-08-07 18:37:31 on dist-test-slave-6k22"
server_key: "e9b74cf80e30ab463028b9aa6f4b0b71"
server_key_iv: "98a67577ac65292bc2f02f92ae452b75"
server_key_version: "encryptionkey@0"
I20260807 18:37:31.713861 1549 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786127579900523-11321-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786127579900523-11321-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786127579900523-11321-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:37:31.718575 1549 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:37:31.722638 1549 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:37:31.722846 1549 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:37:31.730695 1549 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40745
I20260807 18:37:31.730718 1629 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40745 every 8 connection(s)
I20260807 18:37:31.731804 1549 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786127579900523-11321-0/minicluster-data/master-0/data/info.pb
I20260807 18:37:31.734705 1630 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:37:31.736323 11321 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkz56XN/build/debug/bin/kudu as pid 1549
I20260807 18:37:31.736478 11321 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786127579900523-11321-0/minicluster-data/master-0/wal/instance
I20260807 18:37:31.736755 11321 external_mini_cluster.cc:1428] Setting key c39d66d2241a816c1a0293804561215b
I20260807 18:37:31.741797 1630 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9420fe1ee3094e648e787d7ad1f93ece: Bootstrap starting.
I20260807 18:37:31.742565 1634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:31.738260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51557 (local address 127.0.0.1:40745)
0807 18:37:31.738730 (+ 470us) server_negotiation.cc:206] Beginning negotiation
0807 18:37:31.738751 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0807 18:37:31.738798 (+ 47us) server_negotiation.cc:407] Connection header received
0807 18:37:31.739638 (+ 840us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:37:31.739665 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:37:31.740046 (+ 381us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:37:31.740421 (+ 375us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:37:31.741181 (+ 760us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:37:31.741186 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:37:31.741203 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:37:31.741215 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:37:31.741282 (+ 67us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:37:31.741309 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:37:31.741448 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0807 18:37:31.741518 (+ 70us) server_negotiation.cc:299] Negotiation successful
0807 18:37:31.741678 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":306,"thread_start_us":116,"threads_started":1}
I20260807 18:37:31.744306 1630 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9420fe1ee3094e648e787d7ad1f93ece: Neither blocks nor log segments found. Creating new log.
I20260807 18:37:31.745148 1630 log.cc:824] T 00000000000000000000000000000000 P 9420fe1ee3094e648e787d7ad1f93ece: Log is configured to *not* fsync() on all Append() calls
I20260807 18:37:31.745455 11321 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 18:37:31.747628 1630 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9420fe1ee3094e648e787d7ad1f93ece: No bootstrap required, opened a new log
I20260807 18:37:31.749400 1634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:31.747757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39373 (local address 127.0.0.1:40745)
0807 18:37:31.747890 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0807 18:37:31.747895 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:37:31.748069 (+ 174us) server_negotiation.cc:407] Connection header received
0807 18:37:31.748164 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:37:31.748167 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:37:31.748202 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:37:31.748279 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:37:31.749042 (+ 763us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:37:31.749046 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:37:31.749049 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:37:31.749060 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:37:31.749087 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:37:31.749099 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:37:31.749191 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0807 18:37:31.749247 (+ 56us) server_negotiation.cc:299] Negotiation successful
0807 18:37:31.749283 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260807 18:37:31.751978 1630 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9420fe1ee3094e648e787d7ad1f93ece [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9420fe1ee3094e648e787d7ad1f93ece" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40745 } }
I20260807 18:37:31.752239 1630 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9420fe1ee3094e648e787d7ad1f93ece [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:37:31.752287 1630 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9420fe1ee3094e648e787d7ad1f93ece [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9420fe1ee3094e648e787d7ad1f93ece, State: Initialized, Role: FOLLOWER
I20260807 18:37:31.752782 1630 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9420fe1ee3094e648e787d7ad1f93ece [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9420fe1ee3094e648e787d7ad1f93ece" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40745 } }
I20260807 18:37:31.752964 1630 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9420fe1ee3094e648e787d7ad1f93ece [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:37:31.753037 1630 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9420fe1ee3094e648e787d7ad1f93ece [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:37:31.753214 1630 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9420fe1ee3094e648e787d7ad1f93ece [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:37:31.754249 1630 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9420fe1ee3094e648e787d7ad1f93ece [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9420fe1ee3094e648e787d7ad1f93ece" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40745 } }
I20260807 18:37:31.754603 1630 leader_election.cc:302] T 00000000000000000000000000000000 P 9420fe1ee3094e648e787d7ad1f93ece [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9420fe1ee3094e648e787d7ad1f93ece; no voters:
I20260807 18:37:31.754937 1630 leader_election.cc:288] T 00000000000000000000000000000000 P 9420fe1ee3094e648e787d7ad1f93ece [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:37:31.755071 1641 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9420fe1ee3094e648e787d7ad1f93ece [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:37:31.755309 1641 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9420fe1ee3094e648e787d7ad1f93ece [term 1 LEADER]: Becoming Leader. State: Replica: 9420fe1ee3094e648e787d7ad1f93ece, State: Running, Role: LEADER
I20260807 18:37:31.756019 1641 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9420fe1ee3094e648e787d7ad1f93ece [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9420fe1ee3094e648e787d7ad1f93ece" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40745 } }
I20260807 18:37:31.756330 1630 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9420fe1ee3094e648e787d7ad1f93ece [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:37:31.757814 1642 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9420fe1ee3094e648e787d7ad1f93ece [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9420fe1ee3094e648e787d7ad1f93ece" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9420fe1ee3094e648e787d7ad1f93ece" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40745 } } }
I20260807 18:37:31.757862 1643 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9420fe1ee3094e648e787d7ad1f93ece [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9420fe1ee3094e648e787d7ad1f93ece. Latest consensus state: current_term: 1 leader_uuid: "9420fe1ee3094e648e787d7ad1f93ece" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9420fe1ee3094e648e787d7ad1f93ece" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40745 } } }
I20260807 18:37:31.757947 1642 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9420fe1ee3094e648e787d7ad1f93ece [sys.catalog]: This master's current role is: LEADER
I20260807 18:37:31.757987 1643 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9420fe1ee3094e648e787d7ad1f93ece [sys.catalog]: This master's current role is: LEADER
I20260807 18:37:31.758332 1654 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:37:31.761521 1654 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:37:31.767364 1654 catalog_manager.cc:1347] Generated new cluster ID: 76434c0a55eb411c9d1615f601554a16
I20260807 18:37:31.767441 1654 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:37:31.773885 1654 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:37:31.774197 1654 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:37:31.783227 1654 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9420fe1ee3094e648e787d7ad1f93ece: Generated new TSK 0
I20260807 18:37:31.783953 1654 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:37:31.794716 11321 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkz56XN/build/debug/bin/kudu with pid 1549
2026-08-07T18:37:31Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1515 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786127579900523-11321-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:37:31 dist-test-slave-6k22 krb5kdc[1666](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:37:31 dist-test-slave-6k22 krb5kdc[1666](info): set up 2 sockets
Aug 07 18:37:31 dist-test-slave-6k22 krb5kdc[1666](info): commencing operation
krb5kdc: starting...
W20260807 18:37:32.906893 11321 mini_kdc.cc:121] Time spent starting KDC: real 1.086s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786127579900523-11321-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786127579900523-11321-0/krb5kdc/test-user.keytab.
Aug 07 18:37:32 dist-test-slave-6k22 krb5kdc[1666](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127852, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:37:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:37:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786127579900523-11321-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786127579900523-11321-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 18:37:33.038722 11321 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkz56XN/build/debug/bin/kudu
/tmp/dist-test-taskkz56XN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786127579900523-11321-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786127579900523-11321-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786127579900523-11321-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786127579900523-11321-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33857
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786127579900523-11321-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40091
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33857
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786127579900523-11321-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786127579900523-11321-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786127579900523-11321-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:37:33.152864 1679 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:37:33.153187 1679 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:37:33.153260 1679 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:37:33.157305 1679 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:37:33.157398 1679 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:37:33.157436 1679 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:37:33.157495 1679 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:37:33.157532 1679 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:37:33.162914 1679 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40091
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786127579900523-11321-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786127579900523-11321-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33857
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786127579900523-11321-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33857
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786127579900523-11321-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1679
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786127579900523-11321-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:37:33.164029 1679 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:37:33.165222 1679 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:37:33.173938 1684 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:37:33.174081 1679 server_base.cc:1034] running on GCE node
W20260807 18:37:33.173934 1687 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:37:33.174266 1685 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:37:33.175004 1679 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:37:33.176254 1679 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:37:33.177430 1679 hybrid_clock.cc:648] HybridClock initialized: now 1786127853177395 us; error 46 us; skew 500 ppm
Aug 07 18:37:33 dist-test-slave-6k22 krb5kdc[1666](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127853, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:37:33.188265 1679 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 18:37:33.189258 1679 webserver.cc:460] Webserver started at http://127.0.0.1:33925/ using document root <none> and password file <none>
I20260807 18:37:33.189851 1679 fs_manager.cc:359] Metadata directory not provided
I20260807 18:37:33.189946 1679 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:37:33.190209 1679 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:37:33.192415 1679 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786127579900523-11321-0/minicluster-data/master-0/data/instance:
uuid: "1dcc3c6680864948939719e9f2b0535d"
format_stamp: "Formatted at 2026-08-07 18:37:33 on dist-test-slave-6k22"
server_key: "f5f89cad67a4c4364c5e65f3ee0847e7"
server_key_iv: "182f9e2216ed8b558d19c6262ba82300"
server_key_version: "encryptionkey@0"
I20260807 18:37:33.193003 1679 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786127579900523-11321-0/minicluster-data/master-0/wal/instance:
uuid: "1dcc3c6680864948939719e9f2b0535d"
format_stamp: "Formatted at 2026-08-07 18:37:33 on dist-test-slave-6k22"
server_key: "f5f89cad67a4c4364c5e65f3ee0847e7"
server_key_iv: "182f9e2216ed8b558d19c6262ba82300"
server_key_version: "encryptionkey@0"
I20260807 18:37:33.196872 1679 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 18:37:33.199425 1696 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:37:33.200853 1679 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:37:33.201037 1679 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786127579900523-11321-0/minicluster-data/master-0/data,/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786127579900523-11321-0/minicluster-data/master-0/wal
uuid: "1dcc3c6680864948939719e9f2b0535d"
format_stamp: "Formatted at 2026-08-07 18:37:33 on dist-test-slave-6k22"
server_key: "f5f89cad67a4c4364c5e65f3ee0847e7"
server_key_iv: "182f9e2216ed8b558d19c6262ba82300"
server_key_version: "encryptionkey@0"
I20260807 18:37:33.201176 1679 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786127579900523-11321-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786127579900523-11321-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786127579900523-11321-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:37:33.214082 1679 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:37:33.217729 1679 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:37:33.218091 1679 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:37:33.226039 1679 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33857
I20260807 18:37:33.226069 1756 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33857 every 8 connection(s)
I20260807 18:37:33.227031 1679 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786127579900523-11321-0/minicluster-data/master-0/data/info.pb
I20260807 18:37:33.230015 1757 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:37:33.235740 1757 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1dcc3c6680864948939719e9f2b0535d: Bootstrap starting.
I20260807 18:37:33.236153 11321 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkz56XN/build/debug/bin/kudu as pid 1679
I20260807 18:37:33.236312 11321 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786127579900523-11321-0/minicluster-data/master-0/wal/instance
I20260807 18:37:33.236611 11321 external_mini_cluster.cc:1428] Setting key dfd2b6874d8eee1c66744fd9c4226dcd
I20260807 18:37:33.238636 1757 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1dcc3c6680864948939719e9f2b0535d: Neither blocks nor log segments found. Creating new log.
I20260807 18:37:33.239403 1757 log.cc:824] T 00000000000000000000000000000000 P 1dcc3c6680864948939719e9f2b0535d: Log is configured to *not* fsync() on all Append() calls
I20260807 18:37:33.241653 1757 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1dcc3c6680864948939719e9f2b0535d: No bootstrap required, opened a new log
I20260807 18:37:33.246023 1757 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1dcc3c6680864948939719e9f2b0535d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1dcc3c6680864948939719e9f2b0535d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33857 } }
I20260807 18:37:33.246306 1757 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1dcc3c6680864948939719e9f2b0535d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:37:33.246393 1757 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1dcc3c6680864948939719e9f2b0535d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1dcc3c6680864948939719e9f2b0535d, State: Initialized, Role: FOLLOWER
Aug 07 18:37:33 dist-test-slave-6k22 krb5kdc[1666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127852, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260807 18:37:33.246860 1757 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1dcc3c6680864948939719e9f2b0535d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1dcc3c6680864948939719e9f2b0535d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33857 } }
I20260807 18:37:33.247021 1757 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1dcc3c6680864948939719e9f2b0535d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:37:33.247134 1757 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1dcc3c6680864948939719e9f2b0535d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:37:33.247258 1757 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1dcc3c6680864948939719e9f2b0535d [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:37:33.248492 1757 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1dcc3c6680864948939719e9f2b0535d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1dcc3c6680864948939719e9f2b0535d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33857 } }
I20260807 18:37:33.248898 1757 leader_election.cc:302] T 00000000000000000000000000000000 P 1dcc3c6680864948939719e9f2b0535d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1dcc3c6680864948939719e9f2b0535d; no voters:
I20260807 18:37:33.249284 1757 leader_election.cc:288] T 00000000000000000000000000000000 P 1dcc3c6680864948939719e9f2b0535d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:37:33.249452 1763 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1dcc3c6680864948939719e9f2b0535d [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:37:33.249821 1763 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1dcc3c6680864948939719e9f2b0535d [term 1 LEADER]: Becoming Leader. State: Replica: 1dcc3c6680864948939719e9f2b0535d, State: Running, Role: LEADER
I20260807 18:37:33.250267 1763 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1dcc3c6680864948939719e9f2b0535d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1dcc3c6680864948939719e9f2b0535d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33857 } }
I20260807 18:37:33.250602 1757 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1dcc3c6680864948939719e9f2b0535d [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:37:33.251639 1763 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1dcc3c6680864948939719e9f2b0535d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1dcc3c6680864948939719e9f2b0535d. Latest consensus state: current_term: 1 leader_uuid: "1dcc3c6680864948939719e9f2b0535d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1dcc3c6680864948939719e9f2b0535d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33857 } } }
I20260807 18:37:33.251849 1763 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1dcc3c6680864948939719e9f2b0535d [sys.catalog]: This master's current role is: LEADER
I20260807 18:37:33.252329 1764 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1dcc3c6680864948939719e9f2b0535d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1dcc3c6680864948939719e9f2b0535d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1dcc3c6680864948939719e9f2b0535d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33857 } } }
I20260807 18:37:33.252442 1764 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1dcc3c6680864948939719e9f2b0535d [sys.catalog]: This master's current role is: LEADER
I20260807 18:37:33.252580 1772 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:37:33.254307 1760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:33.238133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.223:47943 (local address 127.0.0.1:33857)
0807 18:37:33.238627 (+ 494us) server_negotiation.cc:206] Beginning negotiation
0807 18:37:33.238645 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0807 18:37:33.238709 (+ 64us) server_negotiation.cc:407] Connection header received
0807 18:37:33.239509 (+ 800us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:37:33.239539 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:37:33.239997 (+ 458us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:37:33.240311 (+ 314us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:37:33.241210 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:33.242404 (+ 1194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:33.243666 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:33.244206 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:33.244335 (+ 129us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:33.247704 (+ 3369us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:37:33.247710 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:37:33.247727 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:37:33.247759 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:37:33.249879 (+ 2120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:37:33.250577 (+ 698us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:37:33.250583 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:37:33.250589 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:37:33.250810 (+ 221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:37:33.251262 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:37:33.251268 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:37:33.251272 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:37:33.251852 (+ 580us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:37:33.252680 (+ 828us) server_negotiation.cc:1035] Waiting for connection context
0807 18:37:33.252911 (+ 231us) server_negotiation.cc:299] Negotiation successful
0807 18:37:33.253185 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":318,"thread_start_us":163,"threads_started":1}
I20260807 18:37:33.256982 1772 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:37:33.257541 11321 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 18:37:33.264261 1772 catalog_manager.cc:1347] Generated new cluster ID: 844fbec714d6445b8206085a3d00f415
I20260807 18:37:33.264333 1772 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:37:33.268978 1760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:33.259819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.223:39479 (local address 127.0.0.1:33857)
0807 18:37:33.259948 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0807 18:37:33.259952 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:37:33.260132 (+ 180us) server_negotiation.cc:407] Connection header received
0807 18:37:33.260294 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:37:33.260297 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:37:33.260343 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:37:33.260430 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:37:33.261125 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:33.261752 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:33.262837 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:33.263081 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:33.263202 (+ 121us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:33.264842 (+ 1640us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:37:33.264846 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:37:33.264851 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:37:33.264883 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:37:33.266707 (+ 1824us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:37:33.267243 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:37:33.267247 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:37:33.267249 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:37:33.267345 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:37:33.267770 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:37:33.267774 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:37:33.267776 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:37:33.268016 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:37:33.268189 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0807 18:37:33.268495 (+ 306us) server_negotiation.cc:299] Negotiation successful
0807 18:37:33.268809 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260807 18:37:33.269974 1772 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:37:33.271195 1772 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:37:33.276906 1772 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1dcc3c6680864948939719e9f2b0535d: Generated new TSK 0
I20260807 18:37:33.277669 1772 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:37:33.284879 11321 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkz56XN/build/debug/bin/kudu with pid 1679
W20260807 18:37:33.294586 1663 connection.cc:535] client connection to 127.0.0.1:33857 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:33857): Connection reset by peer (error 104)
2026-08-07T18:37:33Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1489 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786127579900523-11321-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:37:33 dist-test-slave-6k22 krb5kdc[1789](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:37:33 dist-test-slave-6k22 krb5kdc[1789](info): set up 2 sockets
Aug 07 18:37:33 dist-test-slave-6k22 krb5kdc[1789](info): commencing operation
krb5kdc: starting...
W20260807 18:37:34.484038 11321 mini_kdc.cc:121] Time spent starting KDC: real 1.175s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786127579900523-11321-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786127579900523-11321-0/krb5kdc/test-user.keytab.
Aug 07 18:37:34 dist-test-slave-6k22 krb5kdc[1789](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127854, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:37:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:37:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786127579900523-11321-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786127579900523-11321-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 18:37:34.619632 11321 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkz56XN/build/debug/bin/kudu
/tmp/dist-test-taskkz56XN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786127579900523-11321-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786127579900523-11321-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786127579900523-11321-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786127579900523-11321-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37719
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786127579900523-11321-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35831
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37719
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786127579900523-11321-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786127579900523-11321-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786127579900523-11321-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:37:34.741539 1803 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:37:34.741899 1803 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:37:34.742026 1803 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:37:34.746271 1803 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:37:34.746385 1803 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:37:34.746438 1803 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:37:34.746480 1803 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:37:34.746523 1803 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:37:34.752178 1803 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35831
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786127579900523-11321-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786127579900523-11321-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37719
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786127579900523-11321-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37719
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786127579900523-11321-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1803
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786127579900523-11321-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:37:34.753352 1803 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:37:34.754475 1803 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:37:34.764096 1810 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:37:34.764106 1809 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:37:34.764281 1803 server_base.cc:1034] running on GCE node
W20260807 18:37:34.764106 1812 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:37:34.764954 1803 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:37:34.766170 1803 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:37:34.767395 1803 hybrid_clock.cc:648] HybridClock initialized: now 1786127854767372 us; error 37 us; skew 500 ppm
Aug 07 18:37:34 dist-test-slave-6k22 krb5kdc[1789](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127854, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:37:34.770704 1803 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 18:37:34.772053 1803 webserver.cc:460] Webserver started at http://127.0.0.1:45691/ using document root <none> and password file <none>
I20260807 18:37:34.772652 1803 fs_manager.cc:359] Metadata directory not provided
I20260807 18:37:34.772768 1803 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:37:34.773020 1803 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:37:34.775134 1803 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786127579900523-11321-0/minicluster-data/master-0/data/instance:
uuid: "840609018daf4743ab21d94dd6754247"
format_stamp: "Formatted at 2026-08-07 18:37:34 on dist-test-slave-6k22"
server_key: "447f46564e4a6169dff7da600f0b69fb"
server_key_iv: "b9239631245f7dd4c6bbfbc3deb8ed91"
server_key_version: "encryptionkey@0"
I20260807 18:37:34.775794 1803 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786127579900523-11321-0/minicluster-data/master-0/wal/instance:
uuid: "840609018daf4743ab21d94dd6754247"
format_stamp: "Formatted at 2026-08-07 18:37:34 on dist-test-slave-6k22"
server_key: "447f46564e4a6169dff7da600f0b69fb"
server_key_iv: "b9239631245f7dd4c6bbfbc3deb8ed91"
server_key_version: "encryptionkey@0"
I20260807 18:37:34.779532 1803 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260807 18:37:34.782207 1821 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:37:34.783473 1803 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 18:37:34.783653 1803 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786127579900523-11321-0/minicluster-data/master-0/data,/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786127579900523-11321-0/minicluster-data/master-0/wal
uuid: "840609018daf4743ab21d94dd6754247"
format_stamp: "Formatted at 2026-08-07 18:37:34 on dist-test-slave-6k22"
server_key: "447f46564e4a6169dff7da600f0b69fb"
server_key_iv: "b9239631245f7dd4c6bbfbc3deb8ed91"
server_key_version: "encryptionkey@0"
I20260807 18:37:34.783779 1803 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786127579900523-11321-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786127579900523-11321-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786127579900523-11321-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:37:34.799459 1803 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:37:34.802834 1803 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:37:34.803092 1803 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:37:34.811199 1803 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37719
I20260807 18:37:34.811231 1878 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37719 every 8 connection(s)
I20260807 18:37:34.812287 1803 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786127579900523-11321-0/minicluster-data/master-0/data/info.pb
I20260807 18:37:34.815128 1879 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:37:34.817425 11321 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkz56XN/build/debug/bin/kudu as pid 1803
I20260807 18:37:34.817600 11321 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786127579900523-11321-0/minicluster-data/master-0/wal/instance
I20260807 18:37:34.817898 11321 external_mini_cluster.cc:1428] Setting key 6e556c7c64604b43f5ddf04a252143d1
I20260807 18:37:34.821815 1879 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 840609018daf4743ab21d94dd6754247: Bootstrap starting.
I20260807 18:37:34.824524 1879 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 840609018daf4743ab21d94dd6754247: Neither blocks nor log segments found. Creating new log.
I20260807 18:37:34.825302 1879 log.cc:824] T 00000000000000000000000000000000 P 840609018daf4743ab21d94dd6754247: Log is configured to *not* fsync() on all Append() calls
I20260807 18:37:34.827862 1879 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 840609018daf4743ab21d94dd6754247: No bootstrap required, opened a new log
I20260807 18:37:34.828790 1883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:34.819437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.223:50257 (local address 127.0.0.1:37719)
0807 18:37:34.819967 (+ 530us) server_negotiation.cc:206] Beginning negotiation
0807 18:37:34.819987 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0807 18:37:34.820031 (+ 44us) server_negotiation.cc:407] Connection header received
0807 18:37:34.820861 (+ 830us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:37:34.820886 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:37:34.821359 (+ 473us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:37:34.821727 (+ 368us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:37:34.822152 (+ 425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:34.823680 (+ 1528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:34.824841 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:34.825392 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:34.825585 (+ 193us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:34.826555 (+ 970us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:37:34.826562 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:37:34.826580 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:37:34.827132 (+ 552us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:37:34.827223 (+ 91us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:37:34.827256 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:37:34.827410 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0807 18:37:34.827602 (+ 192us) server_negotiation.cc:299] Negotiation successful
0807 18:37:34.827767 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":375,"thread_start_us":141,"threads_started":1}
I20260807 18:37:34.831957 11321 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 18:37:34.832649 1879 raft_consensus.cc:348] T 00000000000000000000000000000000 P 840609018daf4743ab21d94dd6754247 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "840609018daf4743ab21d94dd6754247" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37719 } }
I20260807 18:37:34.832922 1879 raft_consensus.cc:374] T 00000000000000000000000000000000 P 840609018daf4743ab21d94dd6754247 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:37:34.833022 1879 raft_consensus.cc:729] T 00000000000000000000000000000000 P 840609018daf4743ab21d94dd6754247 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 840609018daf4743ab21d94dd6754247, State: Initialized, Role: FOLLOWER
I20260807 18:37:34.833535 1879 consensus_queue.cc:260] T 00000000000000000000000000000000 P 840609018daf4743ab21d94dd6754247 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "840609018daf4743ab21d94dd6754247" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37719 } }
I20260807 18:37:34.833714 1879 raft_consensus.cc:388] T 00000000000000000000000000000000 P 840609018daf4743ab21d94dd6754247 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:37:34.833791 1879 raft_consensus.cc:482] T 00000000000000000000000000000000 P 840609018daf4743ab21d94dd6754247 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:37:34.833902 1879 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 840609018daf4743ab21d94dd6754247 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:37:34.835319 1879 raft_consensus.cc:504] T 00000000000000000000000000000000 P 840609018daf4743ab21d94dd6754247 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "840609018daf4743ab21d94dd6754247" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37719 } }
I20260807 18:37:34.835748 1879 leader_election.cc:302] T 00000000000000000000000000000000 P 840609018daf4743ab21d94dd6754247 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 840609018daf4743ab21d94dd6754247; no voters:
I20260807 18:37:34.836130 1879 leader_election.cc:288] T 00000000000000000000000000000000 P 840609018daf4743ab21d94dd6754247 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:37:34.836244 1890 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 840609018daf4743ab21d94dd6754247 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:37:34.836464 1890 raft_consensus.cc:686] T 00000000000000000000000000000000 P 840609018daf4743ab21d94dd6754247 [term 1 LEADER]: Becoming Leader. State: Replica: 840609018daf4743ab21d94dd6754247, State: Running, Role: LEADER
I20260807 18:37:34.836915 1890 consensus_queue.cc:237] T 00000000000000000000000000000000 P 840609018daf4743ab21d94dd6754247 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "840609018daf4743ab21d94dd6754247" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37719 } }
I20260807 18:37:34.837520 1879 sys_catalog.cc:564] T 00000000000000000000000000000000 P 840609018daf4743ab21d94dd6754247 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:37:34.838248 1892 sys_catalog.cc:455] T 00000000000000000000000000000000 P 840609018daf4743ab21d94dd6754247 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 840609018daf4743ab21d94dd6754247. Latest consensus state: current_term: 1 leader_uuid: "840609018daf4743ab21d94dd6754247" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "840609018daf4743ab21d94dd6754247" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37719 } } }
I20260807 18:37:34.838404 1892 sys_catalog.cc:458] T 00000000000000000000000000000000 P 840609018daf4743ab21d94dd6754247 [sys.catalog]: This master's current role is: LEADER
I20260807 18:37:34.839533 1883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:34.834069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.223:57273 (local address 127.0.0.1:37719)
0807 18:37:34.834212 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0807 18:37:34.834216 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:37:34.834363 (+ 147us) server_negotiation.cc:407] Connection header received
0807 18:37:34.834434 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:37:34.834437 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:37:34.834484 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:37:34.834577 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:37:34.834823 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:34.835736 (+ 913us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:34.836875 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:34.837232 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:34.837310 (+ 78us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:34.838492 (+ 1182us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:37:34.838499 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:37:34.838504 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:37:34.838519 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:37:34.838636 (+ 117us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:37:34.838653 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:37:34.838750 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0807 18:37:34.839141 (+ 391us) server_negotiation.cc:299] Negotiation successful
0807 18:37:34.839185 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260807 18:37:34.839174 1891 sys_catalog.cc:455] T 00000000000000000000000000000000 P 840609018daf4743ab21d94dd6754247 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "840609018daf4743ab21d94dd6754247" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "840609018daf4743ab21d94dd6754247" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37719 } } }
I20260807 18:37:34.840471 1891 sys_catalog.cc:458] T 00000000000000000000000000000000 P 840609018daf4743ab21d94dd6754247 [sys.catalog]: This master's current role is: LEADER
W20260807 18:37:34.843464 1906 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 840609018daf4743ab21d94dd6754247: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 18:37:34.843585 1906 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 18:37:34.843667 1896 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:37:34.844408 1896 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:37:34.850049 1896 catalog_manager.cc:1347] Generated new cluster ID: c85772edc4074d2f9e95ab26ac2e85d0
I20260807 18:37:34.850124 1896 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:37:34.869408 1896 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:37:34.870503 1896 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:37:34.877135 1896 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 840609018daf4743ab21d94dd6754247: Generated new TSK 0
I20260807 18:37:34.877991 1896 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:37:34.890322 11321 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkz56XN/build/debug/bin/kudu with pid 1803
W20260807 18:37:34.899389 1787 connection.cc:535] client connection to 127.0.0.1:37719 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:37719): Connection reset by peer (error 104)
2026-08-07T18:37:34Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1604 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786127579900523-11321-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:37:34 dist-test-slave-6k22 krb5kdc[1912](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:37:34 dist-test-slave-6k22 krb5kdc[1912](info): set up 2 sockets
Aug 07 18:37:34 dist-test-slave-6k22 krb5kdc[1912](info): commencing operation
krb5kdc: starting...
W20260807 18:37:36.041931 11321 mini_kdc.cc:121] Time spent starting KDC: real 1.128s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786127579900523-11321-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786127579900523-11321-0/krb5kdc/test-user.keytab.
Aug 07 18:37:36 dist-test-slave-6k22 krb5kdc[1912](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127856, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:37:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:37:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786127579900523-11321-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786127579900523-11321-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 18:37:36.176550 11321 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkz56XN/build/debug/bin/kudu
/tmp/dist-test-taskkz56XN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786127579900523-11321-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786127579900523-11321-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786127579900523-11321-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786127579900523-11321-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35057
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786127579900523-11321-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43737
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35057
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786127579900523-11321-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786127579900523-11321-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786127579900523-11321-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:37:36.289966 1927 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:37:36.290234 1927 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:37:36.290274 1927 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:37:36.294377 1927 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:37:36.294441 1927 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:37:36.294463 1927 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:37:36.294479 1927 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:37:36.294494 1927 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:37:36.299656 1927 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43737
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786127579900523-11321-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786127579900523-11321-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35057
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786127579900523-11321-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35057
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786127579900523-11321-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1927
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786127579900523-11321-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:37:36.300633 1927 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:37:36.301597 1927 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:37:36.310331 1935 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:37:36.310390 1932 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:37:36.310379 1933 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:37:36.310542 1927 server_base.cc:1034] running on GCE node
I20260807 18:37:36.311156 1927 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:37:36.312350 1927 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:37:36.313549 1927 hybrid_clock.cc:648] HybridClock initialized: now 1786127856313528 us; error 30 us; skew 500 ppm
Aug 07 18:37:36 dist-test-slave-6k22 krb5kdc[1912](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127856, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:37:36.316668 1927 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 18:37:36.317902 1927 webserver.cc:460] Webserver started at http://127.0.0.1:43163/ using document root <none> and password file <none>
I20260807 18:37:36.318642 1927 fs_manager.cc:359] Metadata directory not provided
I20260807 18:37:36.318708 1927 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:37:36.318943 1927 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:37:36.321077 1927 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786127579900523-11321-0/minicluster-data/master-0/data/instance:
uuid: "97febc5f07624480874ab2fe1ee3d3a9"
format_stamp: "Formatted at 2026-08-07 18:37:36 on dist-test-slave-6k22"
server_key: "3e4e5574ccf191d7a9d30d600386429c"
server_key_iv: "d7bf2889fbe7cb75169e9e78aaeecd98"
server_key_version: "encryptionkey@0"
I20260807 18:37:36.321645 1927 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786127579900523-11321-0/minicluster-data/master-0/wal/instance:
uuid: "97febc5f07624480874ab2fe1ee3d3a9"
format_stamp: "Formatted at 2026-08-07 18:37:36 on dist-test-slave-6k22"
server_key: "3e4e5574ccf191d7a9d30d600386429c"
server_key_iv: "d7bf2889fbe7cb75169e9e78aaeecd98"
server_key_version: "encryptionkey@0"
I20260807 18:37:36.325572 1927 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 18:37:36.328011 1943 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:37:36.329231 1927 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 18:37:36.329372 1927 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786127579900523-11321-0/minicluster-data/master-0/data,/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786127579900523-11321-0/minicluster-data/master-0/wal
uuid: "97febc5f07624480874ab2fe1ee3d3a9"
format_stamp: "Formatted at 2026-08-07 18:37:36 on dist-test-slave-6k22"
server_key: "3e4e5574ccf191d7a9d30d600386429c"
server_key_iv: "d7bf2889fbe7cb75169e9e78aaeecd98"
server_key_version: "encryptionkey@0"
I20260807 18:37:36.329492 1927 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786127579900523-11321-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786127579900523-11321-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786127579900523-11321-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:37:36.334201 1927 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:37:36.337111 1927 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:37:36.337320 1927 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:37:36.344738 1927 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35057
I20260807 18:37:36.344769 2003 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35057 every 8 connection(s)
I20260807 18:37:36.345670 1927 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786127579900523-11321-0/minicluster-data/master-0/data/info.pb
I20260807 18:37:36.348479 2004 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:37:36.353549 11321 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkz56XN/build/debug/bin/kudu as pid 1927
I20260807 18:37:36.353705 11321 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786127579900523-11321-0/minicluster-data/master-0/wal/instance
I20260807 18:37:36.353969 2004 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 97febc5f07624480874ab2fe1ee3d3a9: Bootstrap starting.
I20260807 18:37:36.353999 11321 external_mini_cluster.cc:1428] Setting key 14647f5ee6dbbbfd83f9274a29ac68b6
I20260807 18:37:36.356423 2004 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 97febc5f07624480874ab2fe1ee3d3a9: Neither blocks nor log segments found. Creating new log.
I20260807 18:37:36.357319 2004 log.cc:824] T 00000000000000000000000000000000 P 97febc5f07624480874ab2fe1ee3d3a9: Log is configured to *not* fsync() on all Append() calls
I20260807 18:37:36.359169 2007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:36.355312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.223:36825 (local address 127.0.0.1:35057)
0807 18:37:36.355759 (+ 447us) server_negotiation.cc:206] Beginning negotiation
0807 18:37:36.355774 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 18:37:36.355810 (+ 36us) server_negotiation.cc:407] Connection header received
0807 18:37:36.356415 (+ 605us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:37:36.356431 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:37:36.356751 (+ 320us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:37:36.357064 (+ 313us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:37:36.357971 (+ 907us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:37:36.357978 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:37:36.357999 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:37:36.358010 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:37:36.358065 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:37:36.358084 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:37:36.358219 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0807 18:37:36.358309 (+ 90us) server_negotiation.cc:299] Negotiation successful
0807 18:37:36.358507 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":286,"thread_start_us":128,"threads_started":1}
I20260807 18:37:36.359488 2004 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 97febc5f07624480874ab2fe1ee3d3a9: No bootstrap required, opened a new log
I20260807 18:37:36.362283 11321 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 18:37:36.363768 2004 raft_consensus.cc:348] T 00000000000000000000000000000000 P 97febc5f07624480874ab2fe1ee3d3a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97febc5f07624480874ab2fe1ee3d3a9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35057 } }
I20260807 18:37:36.364022 2004 raft_consensus.cc:374] T 00000000000000000000000000000000 P 97febc5f07624480874ab2fe1ee3d3a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:37:36.364120 2004 raft_consensus.cc:729] T 00000000000000000000000000000000 P 97febc5f07624480874ab2fe1ee3d3a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97febc5f07624480874ab2fe1ee3d3a9, State: Initialized, Role: FOLLOWER
I20260807 18:37:36.364616 2004 consensus_queue.cc:260] T 00000000000000000000000000000000 P 97febc5f07624480874ab2fe1ee3d3a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97febc5f07624480874ab2fe1ee3d3a9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35057 } }
I20260807 18:37:36.364771 2004 raft_consensus.cc:388] T 00000000000000000000000000000000 P 97febc5f07624480874ab2fe1ee3d3a9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:37:36.364861 2004 raft_consensus.cc:482] T 00000000000000000000000000000000 P 97febc5f07624480874ab2fe1ee3d3a9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:37:36.364955 2004 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 97febc5f07624480874ab2fe1ee3d3a9 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:37:36.366031 2004 raft_consensus.cc:504] T 00000000000000000000000000000000 P 97febc5f07624480874ab2fe1ee3d3a9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97febc5f07624480874ab2fe1ee3d3a9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35057 } }
I20260807 18:37:36.366168 2007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:36.364412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.223:35389 (local address 127.0.0.1:35057)
0807 18:37:36.364522 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0807 18:37:36.364526 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:37:36.364746 (+ 220us) server_negotiation.cc:407] Connection header received
0807 18:37:36.364921 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:37:36.364924 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:37:36.364957 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:37:36.365052 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:37:36.365781 (+ 729us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:37:36.365789 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:37:36.365792 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:37:36.365803 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:37:36.365829 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:37:36.365846 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:37:36.365931 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0807 18:37:36.366013 (+ 82us) server_negotiation.cc:299] Negotiation successful
0807 18:37:36.366052 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260807 18:37:36.366462 2004 leader_election.cc:302] T 00000000000000000000000000000000 P 97febc5f07624480874ab2fe1ee3d3a9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 97febc5f07624480874ab2fe1ee3d3a9; no voters:
I20260807 18:37:36.366853 2004 leader_election.cc:288] T 00000000000000000000000000000000 P 97febc5f07624480874ab2fe1ee3d3a9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:37:36.367040 2014 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 97febc5f07624480874ab2fe1ee3d3a9 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:37:36.367343 2014 raft_consensus.cc:686] T 00000000000000000000000000000000 P 97febc5f07624480874ab2fe1ee3d3a9 [term 1 LEADER]: Becoming Leader. State: Replica: 97febc5f07624480874ab2fe1ee3d3a9, State: Running, Role: LEADER
I20260807 18:37:36.367719 2014 consensus_queue.cc:237] T 00000000000000000000000000000000 P 97febc5f07624480874ab2fe1ee3d3a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97febc5f07624480874ab2fe1ee3d3a9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35057 } }
I20260807 18:37:36.368166 2004 sys_catalog.cc:564] T 00000000000000000000000000000000 P 97febc5f07624480874ab2fe1ee3d3a9 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:37:36.369457 2016 sys_catalog.cc:455] T 00000000000000000000000000000000 P 97febc5f07624480874ab2fe1ee3d3a9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 97febc5f07624480874ab2fe1ee3d3a9. Latest consensus state: current_term: 1 leader_uuid: "97febc5f07624480874ab2fe1ee3d3a9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97febc5f07624480874ab2fe1ee3d3a9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35057 } } }
I20260807 18:37:36.369580 2016 sys_catalog.cc:458] T 00000000000000000000000000000000 P 97febc5f07624480874ab2fe1ee3d3a9 [sys.catalog]: This master's current role is: LEADER
I20260807 18:37:36.369962 2026 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:37:36.369999 2015 sys_catalog.cc:455] T 00000000000000000000000000000000 P 97febc5f07624480874ab2fe1ee3d3a9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "97febc5f07624480874ab2fe1ee3d3a9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97febc5f07624480874ab2fe1ee3d3a9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35057 } } }
I20260807 18:37:36.370114 2015 sys_catalog.cc:458] T 00000000000000000000000000000000 P 97febc5f07624480874ab2fe1ee3d3a9 [sys.catalog]: This master's current role is: LEADER
I20260807 18:37:36.373075 2026 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:37:36.378824 2026 catalog_manager.cc:1347] Generated new cluster ID: 621d5ee4896f41f280abd37936ab9f99
I20260807 18:37:36.378901 2026 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:37:36.403606 2026 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:37:36.403956 2026 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:37:36.410153 2026 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 97febc5f07624480874ab2fe1ee3d3a9: Generated new TSK 0
I20260807 18:37:36.410852 2026 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:37:36.420573 11321 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkz56XN/build/debug/bin/kudu with pid 1927
2026-08-07T18:37:36Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1528 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786127579900523-11321-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:37:36 dist-test-slave-6k22 krb5kdc[2039](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:37:36 dist-test-slave-6k22 krb5kdc[2039](info): set up 2 sockets
Aug 07 18:37:36 dist-test-slave-6k22 krb5kdc[2039](info): commencing operation
krb5kdc: starting...
W20260807 18:37:37.598569 11321 mini_kdc.cc:121] Time spent starting KDC: real 1.155s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786127579900523-11321-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786127579900523-11321-0/krb5kdc/test-user.keytab.
Aug 07 18:37:37 dist-test-slave-6k22 krb5kdc[2039](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127857, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:37:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:37:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786127579900523-11321-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786127579900523-11321-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 18:37:37.730767 11321 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkz56XN/build/debug/bin/kudu
/tmp/dist-test-taskkz56XN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786127579900523-11321-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786127579900523-11321-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786127579900523-11321-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786127579900523-11321-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:32791
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786127579900523-11321-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45627
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:32791
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786127579900523-11321-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786127579900523-11321-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786127579900523-11321-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:37:37.848361 2053 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:37:37.848678 2053 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:37:37.848757 2053 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:37:37.852792 2053 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:37:37.852876 2053 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:37:37.852916 2053 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:37:37.852974 2053 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:37:37.853015 2053 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:37:37.858287 2053 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45627
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786127579900523-11321-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786127579900523-11321-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:32791
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786127579900523-11321-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:32791
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786127579900523-11321-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2053
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786127579900523-11321-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:37:37.859422 2053 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:37:37.860550 2053 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:37:37.869642 2062 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:37:37.869645 2060 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:37:37.869879 2053 server_base.cc:1034] running on GCE node
W20260807 18:37:37.869642 2059 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:37:37.870597 2053 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:37:37.871867 2053 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:37:37.873077 2053 hybrid_clock.cc:648] HybridClock initialized: now 1786127857873057 us; error 33 us; skew 500 ppm
Aug 07 18:37:37 dist-test-slave-6k22 krb5kdc[2039](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127857, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:37:37.876103 2053 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 18:37:37.876998 2053 webserver.cc:460] Webserver started at http://127.0.0.1:38403/ using document root <none> and password file <none>
I20260807 18:37:37.877647 2053 fs_manager.cc:359] Metadata directory not provided
I20260807 18:37:37.877712 2053 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:37:37.877936 2053 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:37:37.880075 2053 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786127579900523-11321-0/minicluster-data/master-0/data/instance:
uuid: "c1f694aba90b49fba070946657a22e33"
format_stamp: "Formatted at 2026-08-07 18:37:37 on dist-test-slave-6k22"
server_key: "8ba489abbcddbfd2f575a9258e5896e6"
server_key_iv: "e5d35fe8885fd4ef8073c7c6bc552c80"
server_key_version: "encryptionkey@0"
I20260807 18:37:37.880650 2053 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786127579900523-11321-0/minicluster-data/master-0/wal/instance:
uuid: "c1f694aba90b49fba070946657a22e33"
format_stamp: "Formatted at 2026-08-07 18:37:37 on dist-test-slave-6k22"
server_key: "8ba489abbcddbfd2f575a9258e5896e6"
server_key_iv: "e5d35fe8885fd4ef8073c7c6bc552c80"
server_key_version: "encryptionkey@0"
I20260807 18:37:37.884572 2053 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260807 18:37:37.887090 2069 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:37:37.888456 2053 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.001s
I20260807 18:37:37.888608 2053 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786127579900523-11321-0/minicluster-data/master-0/data,/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786127579900523-11321-0/minicluster-data/master-0/wal
uuid: "c1f694aba90b49fba070946657a22e33"
format_stamp: "Formatted at 2026-08-07 18:37:37 on dist-test-slave-6k22"
server_key: "8ba489abbcddbfd2f575a9258e5896e6"
server_key_iv: "e5d35fe8885fd4ef8073c7c6bc552c80"
server_key_version: "encryptionkey@0"
I20260807 18:37:37.888731 2053 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786127579900523-11321-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786127579900523-11321-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786127579900523-11321-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:37:37.910794 2053 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:37:37.914296 2053 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:37:37.914522 2053 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:37:37.922446 2053 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:32791
I20260807 18:37:37.922468 2131 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:32791 every 8 connection(s)
I20260807 18:37:37.923522 2053 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786127579900523-11321-0/minicluster-data/master-0/data/info.pb
I20260807 18:37:37.926473 2132 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:37:37.928344 11321 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkz56XN/build/debug/bin/kudu as pid 2053
I20260807 18:37:37.928489 11321 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786127579900523-11321-0/minicluster-data/master-0/wal/instance
I20260807 18:37:37.928758 11321 external_mini_cluster.cc:1428] Setting key a18ea38196f795f8df5f830fa472bccc
I20260807 18:37:37.932955 2132 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c1f694aba90b49fba070946657a22e33: Bootstrap starting.
I20260807 18:37:37.935272 2132 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c1f694aba90b49fba070946657a22e33: Neither blocks nor log segments found. Creating new log.
I20260807 18:37:37.936249 2132 log.cc:824] T 00000000000000000000000000000000 P c1f694aba90b49fba070946657a22e33: Log is configured to *not* fsync() on all Append() calls
Aug 07 18:37:37 dist-test-slave-6k22 krb5kdc[2039](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127857, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260807 18:37:37.938272 2132 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c1f694aba90b49fba070946657a22e33: No bootstrap required, opened a new log
I20260807 18:37:37.941653 2132 raft_consensus.cc:348] T 00000000000000000000000000000000 P c1f694aba90b49fba070946657a22e33 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1f694aba90b49fba070946657a22e33" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32791 } }
I20260807 18:37:37.941905 2132 raft_consensus.cc:374] T 00000000000000000000000000000000 P c1f694aba90b49fba070946657a22e33 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:37:37.941987 2132 raft_consensus.cc:729] T 00000000000000000000000000000000 P c1f694aba90b49fba070946657a22e33 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1f694aba90b49fba070946657a22e33, State: Initialized, Role: FOLLOWER
I20260807 18:37:37.942487 2132 consensus_queue.cc:260] T 00000000000000000000000000000000 P c1f694aba90b49fba070946657a22e33 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1f694aba90b49fba070946657a22e33" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32791 } }
I20260807 18:37:37.942667 2132 raft_consensus.cc:388] T 00000000000000000000000000000000 P c1f694aba90b49fba070946657a22e33 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:37:37.942778 2132 raft_consensus.cc:482] T 00000000000000000000000000000000 P c1f694aba90b49fba070946657a22e33 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:37:37.942886 2132 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c1f694aba90b49fba070946657a22e33 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:37:37.943300 2136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:37.930208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.223:60569 (local address 127.0.0.1:32791)
0807 18:37:37.930656 (+ 448us) server_negotiation.cc:206] Beginning negotiation
0807 18:37:37.930672 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 18:37:37.930709 (+ 37us) server_negotiation.cc:407] Connection header received
0807 18:37:37.931351 (+ 642us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:37:37.931368 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:37:37.931785 (+ 417us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:37:37.932127 (+ 342us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:37:37.932978 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:37.933998 (+ 1020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:37.934888 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:37.935367 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:37.935540 (+ 173us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:37.938304 (+ 2764us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:37:37.938309 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:37:37.938336 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:37:37.938402 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:37:37.940202 (+ 1800us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:37:37.940733 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:37:37.940738 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:37:37.940743 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:37:37.940868 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:37:37.941222 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:37:37.941229 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:37:37.941231 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:37:37.941617 (+ 386us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:37:37.941859 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0807 18:37:37.942165 (+ 306us) server_negotiation.cc:299] Negotiation successful
0807 18:37:37.942392 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":304,"thread_start_us":139,"threads_started":1}
I20260807 18:37:37.943954 2132 raft_consensus.cc:504] T 00000000000000000000000000000000 P c1f694aba90b49fba070946657a22e33 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1f694aba90b49fba070946657a22e33" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32791 } }
I20260807 18:37:37.944299 2132 leader_election.cc:302] T 00000000000000000000000000000000 P c1f694aba90b49fba070946657a22e33 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c1f694aba90b49fba070946657a22e33; no voters:
I20260807 18:37:37.944644 2132 leader_election.cc:288] T 00000000000000000000000000000000 P c1f694aba90b49fba070946657a22e33 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:37:37.944729 2139 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c1f694aba90b49fba070946657a22e33 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:37:37.944937 2139 raft_consensus.cc:686] T 00000000000000000000000000000000 P c1f694aba90b49fba070946657a22e33 [term 1 LEADER]: Becoming Leader. State: Replica: c1f694aba90b49fba070946657a22e33, State: Running, Role: LEADER
I20260807 18:37:37.945246 2139 consensus_queue.cc:237] T 00000000000000000000000000000000 P c1f694aba90b49fba070946657a22e33 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1f694aba90b49fba070946657a22e33" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32791 } }
I20260807 18:37:37.945983 2132 sys_catalog.cc:564] T 00000000000000000000000000000000 P c1f694aba90b49fba070946657a22e33 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:37:37.946630 11321 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 18:37:37.947196 2140 sys_catalog.cc:455] T 00000000000000000000000000000000 P c1f694aba90b49fba070946657a22e33 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c1f694aba90b49fba070946657a22e33. Latest consensus state: current_term: 1 leader_uuid: "c1f694aba90b49fba070946657a22e33" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1f694aba90b49fba070946657a22e33" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32791 } } }
I20260807 18:37:37.947346 2140 sys_catalog.cc:458] T 00000000000000000000000000000000 P c1f694aba90b49fba070946657a22e33 [sys.catalog]: This master's current role is: LEADER
I20260807 18:37:37.947731 2141 sys_catalog.cc:455] T 00000000000000000000000000000000 P c1f694aba90b49fba070946657a22e33 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c1f694aba90b49fba070946657a22e33" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1f694aba90b49fba070946657a22e33" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32791 } } }
I20260807 18:37:37.947815 2149 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:37:37.947829 2141 sys_catalog.cc:458] T 00000000000000000000000000000000 P c1f694aba90b49fba070946657a22e33 [sys.catalog]: This master's current role is: LEADER
I20260807 18:37:37.951439 2149 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:37:37.957756 2149 catalog_manager.cc:1347] Generated new cluster ID: c7eaf452641e46159e917a46b6a20a25
I20260807 18:37:37.957875 2149 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:37:37.959272 2136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:37.949061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.223:34007 (local address 127.0.0.1:32791)
0807 18:37:37.949164 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0807 18:37:37.949168 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:37:37.950190 (+ 1022us) server_negotiation.cc:407] Connection header received
0807 18:37:37.950376 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:37:37.950379 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:37:37.950427 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:37:37.950482 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:37:37.951412 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:37.952223 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:37.953367 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:37.953592 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:37.953718 (+ 126us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:37.955089 (+ 1371us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:37:37.955093 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:37:37.955095 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:37:37.955119 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:37:37.956788 (+ 1669us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:37:37.957380 (+ 592us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:37:37.957384 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:37:37.957386 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:37:37.957475 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:37:37.957889 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:37:37.957893 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:37:37.957895 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:37:37.958112 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:37:37.958258 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0807 18:37:37.959017 (+ 759us) server_negotiation.cc:299] Negotiation successful
0807 18:37:37.959116 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260807 18:37:37.969964 2149 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:37:37.971300 2149 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:37:37.976505 2149 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c1f694aba90b49fba070946657a22e33: Generated new TSK 0
I20260807 18:37:37.977300 2149 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:37:37.981546 11321 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkz56XN/build/debug/bin/kudu with pid 2053
W20260807 18:37:37.990262 2037 connection.cc:535] client connection to 127.0.0.1:32791 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:32791): Connection reset by peer (error 104)
2026-08-07T18:37:37Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1561 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786127579900523-11321-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:37:38 dist-test-slave-6k22 krb5kdc[2168](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:37:38 dist-test-slave-6k22 krb5kdc[2168](info): set up 2 sockets
Aug 07 18:37:38 dist-test-slave-6k22 krb5kdc[2168](info): commencing operation
krb5kdc: starting...
W20260807 18:37:39.191864 11321 mini_kdc.cc:121] Time spent starting KDC: real 1.187s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786127579900523-11321-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786127579900523-11321-0/krb5kdc/test-user.keytab.
Aug 07 18:37:39 dist-test-slave-6k22 krb5kdc[2168](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127859, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:37:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:37:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786127579900523-11321-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786127579900523-11321-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 18:37:39.323233 11321 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkz56XN/build/debug/bin/kudu
/tmp/dist-test-taskkz56XN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786127579900523-11321-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786127579900523-11321-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786127579900523-11321-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786127579900523-11321-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44259
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786127579900523-11321-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43195
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44259
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786127579900523-11321-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786127579900523-11321-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786127579900523-11321-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:37:39.436815 2181 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:37:39.437163 2181 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:37:39.437206 2181 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:37:39.441112 2181 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:37:39.441169 2181 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:37:39.441190 2181 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:37:39.441205 2181 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:37:39.441221 2181 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:37:39.446406 2181 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43195
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786127579900523-11321-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786127579900523-11321-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44259
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786127579900523-11321-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44259
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786127579900523-11321-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2181
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786127579900523-11321-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:37:39.447496 2181 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:37:39.448580 2181 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:37:39.457726 2189 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:37:39.457726 2186 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:37:39.457773 2187 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:37:39.458063 2181 server_base.cc:1034] running on GCE node
I20260807 18:37:39.458622 2181 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:37:39.459903 2181 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:37:39.461089 2181 hybrid_clock.cc:648] HybridClock initialized: now 1786127859461060 us; error 39 us; skew 500 ppm
Aug 07 18:37:39 dist-test-slave-6k22 krb5kdc[2168](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127859, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:37:39.471491 2181 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 18:37:39.472651 2181 webserver.cc:460] Webserver started at http://127.0.0.1:35011/ using document root <none> and password file <none>
I20260807 18:37:39.473302 2181 fs_manager.cc:359] Metadata directory not provided
I20260807 18:37:39.473371 2181 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:37:39.473644 2181 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:37:39.476179 2181 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786127579900523-11321-0/minicluster-data/master-0/data/instance:
uuid: "5fd2a0b3b35d4d2984cd1753919d21f7"
format_stamp: "Formatted at 2026-08-07 18:37:39 on dist-test-slave-6k22"
server_key: "8a73caf0b453e10d6e46349255ecc9b6"
server_key_iv: "719db8c81df9af899404230c6b185ffb"
server_key_version: "encryptionkey@0"
I20260807 18:37:39.476886 2181 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786127579900523-11321-0/minicluster-data/master-0/wal/instance:
uuid: "5fd2a0b3b35d4d2984cd1753919d21f7"
format_stamp: "Formatted at 2026-08-07 18:37:39 on dist-test-slave-6k22"
server_key: "8a73caf0b453e10d6e46349255ecc9b6"
server_key_iv: "719db8c81df9af899404230c6b185ffb"
server_key_version: "encryptionkey@0"
I20260807 18:37:39.481760 2181 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 18:37:39.484421 2196 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:37:39.485713 2181 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 18:37:39.485854 2181 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786127579900523-11321-0/minicluster-data/master-0/data,/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786127579900523-11321-0/minicluster-data/master-0/wal
uuid: "5fd2a0b3b35d4d2984cd1753919d21f7"
format_stamp: "Formatted at 2026-08-07 18:37:39 on dist-test-slave-6k22"
server_key: "8a73caf0b453e10d6e46349255ecc9b6"
server_key_iv: "719db8c81df9af899404230c6b185ffb"
server_key_version: "encryptionkey@0"
I20260807 18:37:39.485956 2181 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786127579900523-11321-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786127579900523-11321-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786127579900523-11321-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:37:39.499791 2181 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:37:39.503136 2181 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:37:39.503355 2181 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:37:39.510685 2181 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44259
I20260807 18:37:39.510720 2254 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44259 every 8 connection(s)
I20260807 18:37:39.511675 2181 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786127579900523-11321-0/minicluster-data/master-0/data/info.pb
I20260807 18:37:39.514573 2255 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:37:39.520193 2255 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5fd2a0b3b35d4d2984cd1753919d21f7: Bootstrap starting.
I20260807 18:37:39.520347 11321 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkz56XN/build/debug/bin/kudu as pid 2181
I20260807 18:37:39.520540 11321 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786127579900523-11321-0/minicluster-data/master-0/wal/instance
I20260807 18:37:39.520888 11321 external_mini_cluster.cc:1428] Setting key a059e0da9e79cb27446c1eb87fc6e39c
I20260807 18:37:39.522882 2255 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5fd2a0b3b35d4d2984cd1753919d21f7: Neither blocks nor log segments found. Creating new log.
I20260807 18:37:39.523680 2255 log.cc:824] T 00000000000000000000000000000000 P 5fd2a0b3b35d4d2984cd1753919d21f7: Log is configured to *not* fsync() on all Append() calls
I20260807 18:37:39.525987 2255 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5fd2a0b3b35d4d2984cd1753919d21f7: No bootstrap required, opened a new log
I20260807 18:37:39.530376 2255 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5fd2a0b3b35d4d2984cd1753919d21f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5fd2a0b3b35d4d2984cd1753919d21f7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44259 } }
I20260807 18:37:39.530676 2255 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5fd2a0b3b35d4d2984cd1753919d21f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:37:39.530740 2255 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5fd2a0b3b35d4d2984cd1753919d21f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5fd2a0b3b35d4d2984cd1753919d21f7, State: Initialized, Role: FOLLOWER
I20260807 18:37:39.531092 2258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:39.522405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.223:47753 (local address 127.0.0.1:44259)
0807 18:37:39.522875 (+ 470us) server_negotiation.cc:206] Beginning negotiation
0807 18:37:39.522892 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0807 18:37:39.522927 (+ 35us) server_negotiation.cc:407] Connection header received
0807 18:37:39.523662 (+ 735us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:37:39.523682 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:37:39.524098 (+ 416us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:37:39.524376 (+ 278us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:37:39.524865 (+ 489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:39.526263 (+ 1398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:39.527147 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:39.527901 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:39.528070 (+ 169us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:39.528957 (+ 887us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:37:39.528964 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:37:39.528983 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:37:39.529493 (+ 510us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:37:39.529566 (+ 73us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:37:39.529595 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:37:39.529745 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0807 18:37:39.529888 (+ 143us) server_negotiation.cc:299] Negotiation successful
0807 18:37:39.530023 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":324,"thread_start_us":135,"threads_started":1}
I20260807 18:37:39.531231 2255 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5fd2a0b3b35d4d2984cd1753919d21f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5fd2a0b3b35d4d2984cd1753919d21f7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44259 } }
I20260807 18:37:39.531394 2255 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5fd2a0b3b35d4d2984cd1753919d21f7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:37:39.531466 2255 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5fd2a0b3b35d4d2984cd1753919d21f7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:37:39.531605 2255 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5fd2a0b3b35d4d2984cd1753919d21f7 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:37:39.532670 2255 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5fd2a0b3b35d4d2984cd1753919d21f7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5fd2a0b3b35d4d2984cd1753919d21f7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44259 } }
I20260807 18:37:39.533048 2255 leader_election.cc:302] T 00000000000000000000000000000000 P 5fd2a0b3b35d4d2984cd1753919d21f7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5fd2a0b3b35d4d2984cd1753919d21f7; no voters:
I20260807 18:37:39.533418 2255 leader_election.cc:288] T 00000000000000000000000000000000 P 5fd2a0b3b35d4d2984cd1753919d21f7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:37:39.533480 2261 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5fd2a0b3b35d4d2984cd1753919d21f7 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:37:39.533661 2261 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5fd2a0b3b35d4d2984cd1753919d21f7 [term 1 LEADER]: Becoming Leader. State: Replica: 5fd2a0b3b35d4d2984cd1753919d21f7, State: Running, Role: LEADER
I20260807 18:37:39.533975 2261 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5fd2a0b3b35d4d2984cd1753919d21f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5fd2a0b3b35d4d2984cd1753919d21f7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44259 } }
I20260807 18:37:39.534195 11321 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 18:37:39.534688 2255 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5fd2a0b3b35d4d2984cd1753919d21f7 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:37:39.535210 2263 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5fd2a0b3b35d4d2984cd1753919d21f7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5fd2a0b3b35d4d2984cd1753919d21f7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5fd2a0b3b35d4d2984cd1753919d21f7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44259 } } }
I20260807 18:37:39.535344 2263 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5fd2a0b3b35d4d2984cd1753919d21f7 [sys.catalog]: This master's current role is: LEADER
I20260807 18:37:39.535784 2262 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5fd2a0b3b35d4d2984cd1753919d21f7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5fd2a0b3b35d4d2984cd1753919d21f7. Latest consensus state: current_term: 1 leader_uuid: "5fd2a0b3b35d4d2984cd1753919d21f7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5fd2a0b3b35d4d2984cd1753919d21f7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44259 } } }
I20260807 18:37:39.535878 2262 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5fd2a0b3b35d4d2984cd1753919d21f7 [sys.catalog]: This master's current role is: LEADER
W20260807 18:37:39.540319 2281 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5fd2a0b3b35d4d2984cd1753919d21f7: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 18:37:39.540398 2281 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 18:37:39.540458 2269 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:37:39.541307 2269 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:37:39.541368 2258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:39.536585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.223:48465 (local address 127.0.0.1:44259)
0807 18:37:39.536686 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0807 18:37:39.536690 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:37:39.537763 (+ 1073us) server_negotiation.cc:407] Connection header received
0807 18:37:39.537888 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:37:39.537890 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:37:39.537926 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:37:39.537990 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:37:39.538271 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:39.538778 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:39.539699 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:39.539916 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:39.540028 (+ 112us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:39.540888 (+ 860us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:37:39.540891 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:37:39.540894 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:37:39.540907 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:37:39.540940 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:37:39.540951 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:37:39.541051 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0807 18:37:39.541174 (+ 123us) server_negotiation.cc:299] Negotiation successful
0807 18:37:39.541208 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260807 18:37:39.546883 2269 catalog_manager.cc:1347] Generated new cluster ID: 9f676ac8702d4cd3a4504c01070e9d05
I20260807 18:37:39.546952 2269 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:37:39.586066 2269 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:37:39.587381 2269 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:37:39.596341 2269 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5fd2a0b3b35d4d2984cd1753919d21f7: Generated new TSK 0
I20260807 18:37:39.597046 2269 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:37:39.612928 11321 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkz56XN/build/debug/bin/kudu with pid 2181
W20260807 18:37:39.621923 2165 connection.cc:535] client connection to 127.0.0.1:44259 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:44259): Connection reset by peer (error 104)
2026-08-07T18:37:39Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1631 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786127579900523-11321-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:37:39 dist-test-slave-6k22 krb5kdc[2289](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:37:39 dist-test-slave-6k22 krb5kdc[2289](info): set up 2 sockets
Aug 07 18:37:39 dist-test-slave-6k22 krb5kdc[2289](info): commencing operation
krb5kdc: starting...
W20260807 18:37:40.713829 11321 mini_kdc.cc:121] Time spent starting KDC: real 1.077s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786127579900523-11321-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786127579900523-11321-0/krb5kdc/test-user.keytab.
Aug 07 18:37:40 dist-test-slave-6k22 krb5kdc[2289](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127860, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:37:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:37:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786127579900523-11321-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786127579900523-11321-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 18:37:40.842278 11321 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkz56XN/build/debug/bin/kudu
/tmp/dist-test-taskkz56XN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786127579900523-11321-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786127579900523-11321-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786127579900523-11321-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786127579900523-11321-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36437
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786127579900523-11321-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42439
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36437
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786127579900523-11321-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786127579900523-11321-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786127579900523-11321-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:37:40.956957 2302 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:37:40.957274 2302 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:37:40.957353 2302 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:37:40.961337 2302 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:37:40.961428 2302 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:37:40.961467 2302 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:37:40.961527 2302 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:37:40.961565 2302 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:37:40.966674 2302 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42439
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786127579900523-11321-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786127579900523-11321-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36437
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786127579900523-11321-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36437
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786127579900523-11321-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2302
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786127579900523-11321-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:37:40.967794 2302 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:37:40.968830 2302 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:37:40.977684 2308 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:37:40.977684 2312 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:37:40.977849 2302 server_base.cc:1034] running on GCE node
W20260807 18:37:40.978041 2309 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:37:40.978657 2302 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:37:40.979943 2302 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:37:40.981112 2302 hybrid_clock.cc:648] HybridClock initialized: now 1786127860981090 us; error 32 us; skew 500 ppm
Aug 07 18:37:40 dist-test-slave-6k22 krb5kdc[2289](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127860, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:37:40.983897 2302 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 18:37:40.984787 2302 webserver.cc:460] Webserver started at http://127.0.0.1:45075/ using document root <none> and password file <none>
I20260807 18:37:40.985342 2302 fs_manager.cc:359] Metadata directory not provided
I20260807 18:37:40.985400 2302 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:37:40.985576 2302 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:37:40.987735 2302 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786127579900523-11321-0/minicluster-data/master-0/data/instance:
uuid: "998070c3717946599772440c37862606"
format_stamp: "Formatted at 2026-08-07 18:37:40 on dist-test-slave-6k22"
server_key: "97a30b4f22acc9821ef4404bd7e86f48"
server_key_iv: "0c140cfe3296731e5e46f3f139cdd35f"
server_key_version: "encryptionkey@0"
I20260807 18:37:40.988355 2302 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786127579900523-11321-0/minicluster-data/master-0/wal/instance:
uuid: "998070c3717946599772440c37862606"
format_stamp: "Formatted at 2026-08-07 18:37:40 on dist-test-slave-6k22"
server_key: "97a30b4f22acc9821ef4404bd7e86f48"
server_key_iv: "0c140cfe3296731e5e46f3f139cdd35f"
server_key_version: "encryptionkey@0"
I20260807 18:37:40.992301 2302 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260807 18:37:40.994777 2320 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:37:40.996124 2302 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 18:37:40.996279 2302 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786127579900523-11321-0/minicluster-data/master-0/data,/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786127579900523-11321-0/minicluster-data/master-0/wal
uuid: "998070c3717946599772440c37862606"
format_stamp: "Formatted at 2026-08-07 18:37:40 on dist-test-slave-6k22"
server_key: "97a30b4f22acc9821ef4404bd7e86f48"
server_key_iv: "0c140cfe3296731e5e46f3f139cdd35f"
server_key_version: "encryptionkey@0"
I20260807 18:37:40.996409 2302 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786127579900523-11321-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786127579900523-11321-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786127579900523-11321-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:37:41.001194 2302 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:37:41.004282 2302 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:37:41.004482 2302 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:37:41.011770 2302 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36437
I20260807 18:37:41.011896 2380 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36437 every 8 connection(s)
I20260807 18:37:41.012758 2302 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786127579900523-11321-0/minicluster-data/master-0/data/info.pb
I20260807 18:37:41.015610 2381 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:37:41.018905 11321 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkz56XN/build/debug/bin/kudu as pid 2302
I20260807 18:37:41.019069 11321 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786127579900523-11321-0/minicluster-data/master-0/wal/instance
I20260807 18:37:41.019363 11321 external_mini_cluster.cc:1428] Setting key bd8921650886e3a834de6a61fdc24562
I20260807 18:37:41.021458 2381 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 998070c3717946599772440c37862606: Bootstrap starting.
I20260807 18:37:41.024109 2381 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 998070c3717946599772440c37862606: Neither blocks nor log segments found. Creating new log.
I20260807 18:37:41.024639 2384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:41.020745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.223:49497 (local address 127.0.0.1:36437)
0807 18:37:41.021151 (+ 406us) server_negotiation.cc:206] Beginning negotiation
0807 18:37:41.021166 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 18:37:41.021203 (+ 37us) server_negotiation.cc:407] Connection header received
0807 18:37:41.021868 (+ 665us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:37:41.021886 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:37:41.022168 (+ 282us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:37:41.022447 (+ 279us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:37:41.023365 (+ 918us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:37:41.023370 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:37:41.023382 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:37:41.023393 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:37:41.023444 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:37:41.023462 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:37:41.023549 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0807 18:37:41.023715 (+ 166us) server_negotiation.cc:299] Negotiation successful
0807 18:37:41.023900 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":244,"thread_start_us":119,"threads_started":1}
I20260807 18:37:41.024926 2381 log.cc:824] T 00000000000000000000000000000000 P 998070c3717946599772440c37862606: Log is configured to *not* fsync() on all Append() calls
I20260807 18:37:41.027002 2381 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 998070c3717946599772440c37862606: No bootstrap required, opened a new log
I20260807 18:37:41.027395 11321 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 18:37:41.031589 2384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:41.029757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.223:49859 (local address 127.0.0.1:36437)
0807 18:37:41.029889 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0807 18:37:41.029893 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:37:41.030104 (+ 211us) server_negotiation.cc:407] Connection header received
0807 18:37:41.030245 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:37:41.030248 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:37:41.030277 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:37:41.030344 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:37:41.031123 (+ 779us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:37:41.031126 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:37:41.031128 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:37:41.031138 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:37:41.031160 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:37:41.031170 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:37:41.031256 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0807 18:37:41.031376 (+ 120us) server_negotiation.cc:299] Negotiation successful
0807 18:37:41.031410 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260807 18:37:41.031939 2381 raft_consensus.cc:348] T 00000000000000000000000000000000 P 998070c3717946599772440c37862606 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "998070c3717946599772440c37862606" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36437 } }
I20260807 18:37:41.032220 2381 raft_consensus.cc:374] T 00000000000000000000000000000000 P 998070c3717946599772440c37862606 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:37:41.032291 2381 raft_consensus.cc:729] T 00000000000000000000000000000000 P 998070c3717946599772440c37862606 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 998070c3717946599772440c37862606, State: Initialized, Role: FOLLOWER
I20260807 18:37:41.032768 2381 consensus_queue.cc:260] T 00000000000000000000000000000000 P 998070c3717946599772440c37862606 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "998070c3717946599772440c37862606" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36437 } }
I20260807 18:37:41.032924 2381 raft_consensus.cc:388] T 00000000000000000000000000000000 P 998070c3717946599772440c37862606 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:37:41.032996 2381 raft_consensus.cc:482] T 00000000000000000000000000000000 P 998070c3717946599772440c37862606 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:37:41.033097 2381 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 998070c3717946599772440c37862606 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:37:41.034024 2381 raft_consensus.cc:504] T 00000000000000000000000000000000 P 998070c3717946599772440c37862606 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "998070c3717946599772440c37862606" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36437 } }
I20260807 18:37:41.034374 2381 leader_election.cc:302] T 00000000000000000000000000000000 P 998070c3717946599772440c37862606 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 998070c3717946599772440c37862606; no voters:
I20260807 18:37:41.034699 2381 leader_election.cc:288] T 00000000000000000000000000000000 P 998070c3717946599772440c37862606 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:37:41.034824 2391 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 998070c3717946599772440c37862606 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:37:41.035053 2391 raft_consensus.cc:686] T 00000000000000000000000000000000 P 998070c3717946599772440c37862606 [term 1 LEADER]: Becoming Leader. State: Replica: 998070c3717946599772440c37862606, State: Running, Role: LEADER
I20260807 18:37:41.035418 2391 consensus_queue.cc:237] T 00000000000000000000000000000000 P 998070c3717946599772440c37862606 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "998070c3717946599772440c37862606" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36437 } }
I20260807 18:37:41.036059 2381 sys_catalog.cc:564] T 00000000000000000000000000000000 P 998070c3717946599772440c37862606 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:37:41.037161 2392 sys_catalog.cc:455] T 00000000000000000000000000000000 P 998070c3717946599772440c37862606 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "998070c3717946599772440c37862606" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "998070c3717946599772440c37862606" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36437 } } }
I20260807 18:37:41.037290 2392 sys_catalog.cc:458] T 00000000000000000000000000000000 P 998070c3717946599772440c37862606 [sys.catalog]: This master's current role is: LEADER
I20260807 18:37:41.037626 2393 sys_catalog.cc:455] T 00000000000000000000000000000000 P 998070c3717946599772440c37862606 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 998070c3717946599772440c37862606. Latest consensus state: current_term: 1 leader_uuid: "998070c3717946599772440c37862606" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "998070c3717946599772440c37862606" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36437 } } }
I20260807 18:37:41.037784 2393 sys_catalog.cc:458] T 00000000000000000000000000000000 P 998070c3717946599772440c37862606 [sys.catalog]: This master's current role is: LEADER
I20260807 18:37:41.038486 2402 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:37:41.041297 2402 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:37:41.046727 2402 catalog_manager.cc:1347] Generated new cluster ID: 119f8e5b05b6453a8bcbd28ea0ceb8f2
I20260807 18:37:41.046798 2402 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:37:41.057214 2402 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:37:41.057497 2402 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:37:41.062743 2402 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 998070c3717946599772440c37862606: Generated new TSK 0
I20260807 18:37:41.063391 2402 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:37:41.066787 11321 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkz56XN/build/debug/bin/kudu with pid 2302
2026-08-07T18:37:41Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1453 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786127579900523-11321-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:37:41 dist-test-slave-6k22 krb5kdc[2415](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:37:41 dist-test-slave-6k22 krb5kdc[2415](info): set up 2 sockets
Aug 07 18:37:41 dist-test-slave-6k22 krb5kdc[2415](info): commencing operation
krb5kdc: starting...
W20260807 18:37:42.165663 11321 mini_kdc.cc:121] Time spent starting KDC: real 1.076s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786127579900523-11321-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786127579900523-11321-0/krb5kdc/test-user.keytab.
Aug 07 18:37:42 dist-test-slave-6k22 krb5kdc[2415](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127862, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:37:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:37:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.14.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.14.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.14.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.126.keytab.
Entry for principal kudu/127.11.14.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.126.keytab.
I20260807 18:37:42.295208 11321 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkz56XN/build/debug/bin/kudu
/tmp/dist-test-taskkz56XN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786127579900523-11321-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786127579900523-11321-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786127579900523-11321-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786127579900523-11321-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.14.126:41713
--webserver_interface=127.11.14.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.126.keytab
--principal=kudu/127.11.14.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.14.84:37259
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.14.126:41713
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786127579900523-11321-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786127579900523-11321-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786127579900523-11321-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:37:42.409897 2432 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:37:42.410207 2432 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:37:42.410286 2432 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:37:42.414229 2432 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:37:42.414330 2432 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:37:42.414371 2432 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:37:42.414408 2432 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:37:42.414444 2432 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:37:42.419701 2432 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.14.84:37259
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786127579900523-11321-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786127579900523-11321-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.14.126:41713
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.126.keytab
--principal=kudu/127.11.14.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.14.126:41713
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786127579900523-11321-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.14.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2432
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786127579900523-11321-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:37:42.420779 2432 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:37:42.421820 2432 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:37:42.430447 2438 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:37:42.430526 2440 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:37:42.430584 2432 server_base.cc:1034] running on GCE node
W20260807 18:37:42.430737 2437 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:37:42.431263 2432 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:37:42.432408 2432 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:37:42.433588 2432 hybrid_clock.cc:648] HybridClock initialized: now 1786127862433560 us; error 40 us; skew 500 ppm
Aug 07 18:37:42 dist-test-slave-6k22 krb5kdc[2415](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127862, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:37:42.436638 2432 init.cc:377] Logged in from keytab as kudu/127.11.14.126@KRBTEST.COM (short username kudu)
I20260807 18:37:42.437603 2432 webserver.cc:460] Webserver started at http://127.11.14.126:34127/ using document root <none> and password file <none>
I20260807 18:37:42.438186 2432 fs_manager.cc:359] Metadata directory not provided
I20260807 18:37:42.438288 2432 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:37:42.438550 2432 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:37:42.440760 2432 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786127579900523-11321-0/minicluster-data/master-0/data/instance:
uuid: "715666b10d76480da4f7bf0dc18ac17f"
format_stamp: "Formatted at 2026-08-07 18:37:42 on dist-test-slave-6k22"
server_key: "b4fb3387f372506547b1244639b5dc51"
server_key_iv: "0351df3df38c971ae7b40eaedbd8b3fb"
server_key_version: "encryptionkey@0"
I20260807 18:37:42.441344 2432 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786127579900523-11321-0/minicluster-data/master-0/wal/instance:
uuid: "715666b10d76480da4f7bf0dc18ac17f"
format_stamp: "Formatted at 2026-08-07 18:37:42 on dist-test-slave-6k22"
server_key: "b4fb3387f372506547b1244639b5dc51"
server_key_iv: "0351df3df38c971ae7b40eaedbd8b3fb"
server_key_version: "encryptionkey@0"
I20260807 18:37:42.445103 2432 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 18:37:42.447451 2448 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:37:42.448689 2432 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 18:37:42.448840 2432 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786127579900523-11321-0/minicluster-data/master-0/data,/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786127579900523-11321-0/minicluster-data/master-0/wal
uuid: "715666b10d76480da4f7bf0dc18ac17f"
format_stamp: "Formatted at 2026-08-07 18:37:42 on dist-test-slave-6k22"
server_key: "b4fb3387f372506547b1244639b5dc51"
server_key_iv: "0351df3df38c971ae7b40eaedbd8b3fb"
server_key_version: "encryptionkey@0"
I20260807 18:37:42.448956 2432 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786127579900523-11321-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786127579900523-11321-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786127579900523-11321-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:37:42.471258 2432 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:37:42.474366 2432 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:37:42.474581 2432 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:37:42.482077 2432 rpc_server.cc:307] RPC server started. Bound to: 127.11.14.126:41713
I20260807 18:37:42.482105 2512 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.14.126:41713 every 8 connection(s)
I20260807 18:37:42.483165 2432 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786127579900523-11321-0/minicluster-data/master-0/data/info.pb
I20260807 18:37:42.485987 2513 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:37:42.491616 2513 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 715666b10d76480da4f7bf0dc18ac17f: Bootstrap starting.
I20260807 18:37:42.492622 11321 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkz56XN/build/debug/bin/kudu as pid 2432
I20260807 18:37:42.492798 11321 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786127579900523-11321-0/minicluster-data/master-0/wal/instance
I20260807 18:37:42.493100 11321 external_mini_cluster.cc:1428] Setting key 9ed119add9587a4f6d9b0e6c139ff67b
I20260807 18:37:42.494586 2513 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 715666b10d76480da4f7bf0dc18ac17f: Neither blocks nor log segments found. Creating new log.
I20260807 18:37:42.495334 2513 log.cc:824] T 00000000000000000000000000000000 P 715666b10d76480da4f7bf0dc18ac17f: Log is configured to *not* fsync() on all Append() calls
I20260807 18:37:42.497332 2513 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 715666b10d76480da4f7bf0dc18ac17f: No bootstrap required, opened a new log
I20260807 18:37:42.500761 2513 raft_consensus.cc:348] T 00000000000000000000000000000000 P 715666b10d76480da4f7bf0dc18ac17f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "715666b10d76480da4f7bf0dc18ac17f" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 41713 } }
I20260807 18:37:42.501046 2513 raft_consensus.cc:374] T 00000000000000000000000000000000 P 715666b10d76480da4f7bf0dc18ac17f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:37:42.501116 2513 raft_consensus.cc:729] T 00000000000000000000000000000000 P 715666b10d76480da4f7bf0dc18ac17f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 715666b10d76480da4f7bf0dc18ac17f, State: Initialized, Role: FOLLOWER
I20260807 18:37:42.501533 2513 consensus_queue.cc:260] T 00000000000000000000000000000000 P 715666b10d76480da4f7bf0dc18ac17f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "715666b10d76480da4f7bf0dc18ac17f" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 41713 } }
I20260807 18:37:42.501655 2513 raft_consensus.cc:388] T 00000000000000000000000000000000 P 715666b10d76480da4f7bf0dc18ac17f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:37:42.501746 2513 raft_consensus.cc:482] T 00000000000000000000000000000000 P 715666b10d76480da4f7bf0dc18ac17f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:37:42.501874 2513 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 715666b10d76480da4f7bf0dc18ac17f [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:37:42.502955 2513 raft_consensus.cc:504] T 00000000000000000000000000000000 P 715666b10d76480da4f7bf0dc18ac17f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "715666b10d76480da4f7bf0dc18ac17f" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 41713 } }
Aug 07 18:37:42 dist-test-slave-6k22 krb5kdc[2415](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127862, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.14.126@KRBTEST.COM
I20260807 18:37:42.503340 2513 leader_election.cc:302] T 00000000000000000000000000000000 P 715666b10d76480da4f7bf0dc18ac17f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 715666b10d76480da4f7bf0dc18ac17f; no voters:
I20260807 18:37:42.503720 2513 leader_election.cc:288] T 00000000000000000000000000000000 P 715666b10d76480da4f7bf0dc18ac17f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:37:42.503826 2520 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 715666b10d76480da4f7bf0dc18ac17f [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:37:42.504148 2520 raft_consensus.cc:686] T 00000000000000000000000000000000 P 715666b10d76480da4f7bf0dc18ac17f [term 1 LEADER]: Becoming Leader. State: Replica: 715666b10d76480da4f7bf0dc18ac17f, State: Running, Role: LEADER
I20260807 18:37:42.504534 2520 consensus_queue.cc:237] T 00000000000000000000000000000000 P 715666b10d76480da4f7bf0dc18ac17f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "715666b10d76480da4f7bf0dc18ac17f" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 41713 } }
I20260807 18:37:42.505163 2513 sys_catalog.cc:564] T 00000000000000000000000000000000 P 715666b10d76480da4f7bf0dc18ac17f [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:37:42.505934 2520 sys_catalog.cc:455] T 00000000000000000000000000000000 P 715666b10d76480da4f7bf0dc18ac17f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 715666b10d76480da4f7bf0dc18ac17f. Latest consensus state: current_term: 1 leader_uuid: "715666b10d76480da4f7bf0dc18ac17f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "715666b10d76480da4f7bf0dc18ac17f" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 41713 } } }
I20260807 18:37:42.506069 2520 sys_catalog.cc:458] T 00000000000000000000000000000000 P 715666b10d76480da4f7bf0dc18ac17f [sys.catalog]: This master's current role is: LEADER
I20260807 18:37:42.506834 2521 sys_catalog.cc:455] T 00000000000000000000000000000000 P 715666b10d76480da4f7bf0dc18ac17f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "715666b10d76480da4f7bf0dc18ac17f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "715666b10d76480da4f7bf0dc18ac17f" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 41713 } } }
I20260807 18:37:42.506958 2521 sys_catalog.cc:458] T 00000000000000000000000000000000 P 715666b10d76480da4f7bf0dc18ac17f [sys.catalog]: This master's current role is: LEADER
I20260807 18:37:42.510298 2518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:42.494285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52775 (local address 127.11.14.126:41713)
0807 18:37:42.494824 (+ 539us) server_negotiation.cc:206] Beginning negotiation
0807 18:37:42.494841 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0807 18:37:42.494878 (+ 37us) server_negotiation.cc:407] Connection header received
0807 18:37:42.495600 (+ 722us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:37:42.495620 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:37:42.496060 (+ 440us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:37:42.496687 (+ 627us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:37:42.497690 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:42.498778 (+ 1088us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:42.499818 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:42.500337 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:42.500533 (+ 196us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:42.503835 (+ 3302us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:37:42.503844 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:37:42.503866 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:37:42.503902 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:37:42.506397 (+ 2495us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:37:42.507216 (+ 819us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:37:42.507224 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:37:42.507231 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:37:42.507386 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:37:42.507915 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:37:42.507921 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:37:42.507924 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:37:42.508467 (+ 543us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:37:42.508667 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0807 18:37:42.508998 (+ 331us) server_negotiation.cc:299] Negotiation successful
0807 18:37:42.509268 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":368,"thread_start_us":151,"threads_started":1}
W20260807 18:37:42.510515 2536 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 715666b10d76480da4f7bf0dc18ac17f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 18:37:42.510581 2536 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 18:37:42.510638 2530 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:37:42.511291 2530 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:37:42.513993 11321 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 18:37:42.516902 2530 catalog_manager.cc:1347] Generated new cluster ID: 77632af838054425923360f9d96a0b91
I20260807 18:37:42.516987 2530 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:37:42.526474 2518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:42.516842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55713 (local address 127.11.14.126:41713)
0807 18:37:42.516956 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0807 18:37:42.516961 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:37:42.517109 (+ 148us) server_negotiation.cc:407] Connection header received
0807 18:37:42.517311 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:37:42.517315 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:37:42.517373 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:37:42.517465 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:37:42.518402 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:42.519248 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:42.520507 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:42.520847 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:42.520943 (+ 96us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:42.522776 (+ 1833us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:37:42.522780 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:37:42.522783 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:37:42.522806 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:37:42.524531 (+ 1725us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:37:42.525113 (+ 582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:37:42.525116 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:37:42.525118 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:37:42.525200 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:37:42.525568 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:37:42.525571 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:37:42.525573 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:37:42.525762 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:37:42.525890 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0807 18:37:42.526204 (+ 314us) server_negotiation.cc:299] Negotiation successful
0807 18:37:42.526302 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260807 18:37:42.534570 2530 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:37:42.535626 2530 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:37:42.539481 2530 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 715666b10d76480da4f7bf0dc18ac17f: Generated new TSK 0
I20260807 18:37:42.540189 2530 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:37:42.547601 11321 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkz56XN/build/debug/bin/kudu with pid 2432
W20260807 18:37:42.556730 2413 connection.cc:535] client connection to 127.11.14.126:41713 recv error: Network error: failed to read from TLS socket (remote: 127.11.14.126:41713): Connection reset by peer (error 104)
2026-08-07T18:37:42Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1481 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786127579900523-11321-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:37:42 dist-test-slave-6k22 krb5kdc[2547](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:37:42 dist-test-slave-6k22 krb5kdc[2547](info): set up 2 sockets
Aug 07 18:37:42 dist-test-slave-6k22 krb5kdc[2547](info): commencing operation
krb5kdc: starting...
W20260807 18:37:43.658267 11321 mini_kdc.cc:121] Time spent starting KDC: real 1.087s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786127579900523-11321-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786127579900523-11321-0/krb5kdc/test-user.keytab.
Aug 07 18:37:43 dist-test-slave-6k22 krb5kdc[2547](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127863, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:37:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:37:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.14.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.14.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.14.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.126.keytab.
Entry for principal kudu/127.11.14.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.126.keytab.
I20260807 18:37:43.787103 11321 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkz56XN/build/debug/bin/kudu
/tmp/dist-test-taskkz56XN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786127579900523-11321-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786127579900523-11321-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786127579900523-11321-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786127579900523-11321-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.14.126:37393
--webserver_interface=127.11.14.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.126.keytab
--principal=kudu/127.11.14.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.14.84:34291
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.14.126:37393
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786127579900523-11321-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786127579900523-11321-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786127579900523-11321-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:37:43.900283 2563 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:37:43.900535 2563 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:37:43.900630 2563 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:37:43.904649 2563 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:37:43.904714 2563 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:37:43.904734 2563 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:37:43.904772 2563 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:37:43.904809 2563 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:37:43.910000 2563 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.14.84:34291
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786127579900523-11321-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786127579900523-11321-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.14.126:37393
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.126.keytab
--principal=kudu/127.11.14.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.14.126:37393
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786127579900523-11321-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.14.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2563
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786127579900523-11321-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:37:43.911064 2563 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:37:43.912150 2563 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:37:43.920954 2572 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:37:43.920918 2569 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:37:43.921022 2563 server_base.cc:1034] running on GCE node
W20260807 18:37:43.920914 2570 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:37:43.921795 2563 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:37:43.922966 2563 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:37:43.924160 2563 hybrid_clock.cc:648] HybridClock initialized: now 1786127863924117 us; error 46 us; skew 500 ppm
Aug 07 18:37:43 dist-test-slave-6k22 krb5kdc[2547](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127863, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:37:43.926970 2563 init.cc:377] Logged in from keytab as kudu/127.11.14.126@KRBTEST.COM (short username kudu)
I20260807 18:37:43.927949 2563 webserver.cc:460] Webserver started at http://127.11.14.126:40907/ using document root <none> and password file <none>
I20260807 18:37:43.928524 2563 fs_manager.cc:359] Metadata directory not provided
I20260807 18:37:43.928615 2563 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:37:43.928865 2563 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:37:43.930953 2563 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786127579900523-11321-0/minicluster-data/master-0/data/instance:
uuid: "c6ef2450d9434c7186797dbde9cbc67c"
format_stamp: "Formatted at 2026-08-07 18:37:43 on dist-test-slave-6k22"
server_key: "0cde90f53c0b213bd99978a01ee798ae"
server_key_iv: "e8060f01861cb696a35ec7c45bf9fbb0"
server_key_version: "encryptionkey@0"
I20260807 18:37:43.931529 2563 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786127579900523-11321-0/minicluster-data/master-0/wal/instance:
uuid: "c6ef2450d9434c7186797dbde9cbc67c"
format_stamp: "Formatted at 2026-08-07 18:37:43 on dist-test-slave-6k22"
server_key: "0cde90f53c0b213bd99978a01ee798ae"
server_key_iv: "e8060f01861cb696a35ec7c45bf9fbb0"
server_key_version: "encryptionkey@0"
I20260807 18:37:43.935669 2563 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 18:37:43.938199 2580 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:37:43.939504 2563 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 18:37:43.939723 2563 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786127579900523-11321-0/minicluster-data/master-0/data,/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786127579900523-11321-0/minicluster-data/master-0/wal
uuid: "c6ef2450d9434c7186797dbde9cbc67c"
format_stamp: "Formatted at 2026-08-07 18:37:43 on dist-test-slave-6k22"
server_key: "0cde90f53c0b213bd99978a01ee798ae"
server_key_iv: "e8060f01861cb696a35ec7c45bf9fbb0"
server_key_version: "encryptionkey@0"
I20260807 18:37:43.939855 2563 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786127579900523-11321-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786127579900523-11321-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786127579900523-11321-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:37:43.970227 2563 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:37:43.973618 2563 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:37:43.973840 2563 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:37:43.981287 2563 rpc_server.cc:307] RPC server started. Bound to: 127.11.14.126:37393
I20260807 18:37:43.981314 2640 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.14.126:37393 every 8 connection(s)
I20260807 18:37:43.982262 2563 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786127579900523-11321-0/minicluster-data/master-0/data/info.pb
I20260807 18:37:43.983382 11321 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkz56XN/build/debug/bin/kudu as pid 2563
I20260807 18:37:43.983525 11321 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786127579900523-11321-0/minicluster-data/master-0/wal/instance
I20260807 18:37:43.983871 11321 external_mini_cluster.cc:1428] Setting key 26f4badf16210b11f3b3528a34cdb284
I20260807 18:37:43.985816 2641 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:37:43.991755 2641 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c6ef2450d9434c7186797dbde9cbc67c: Bootstrap starting.
I20260807 18:37:43.993371 2645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:43.985358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55603 (local address 127.11.14.126:37393)
0807 18:37:43.985849 (+ 491us) server_negotiation.cc:206] Beginning negotiation
0807 18:37:43.985890 (+ 41us) server_negotiation.cc:399] Waiting for connection header
0807 18:37:43.985938 (+ 48us) server_negotiation.cc:407] Connection header received
0807 18:37:43.986720 (+ 782us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:37:43.986740 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:37:43.987093 (+ 353us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:37:43.987445 (+ 352us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:37:43.987886 (+ 441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:43.988888 (+ 1002us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:43.989818 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:43.990385 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:43.990551 (+ 166us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:43.991546 (+ 995us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:37:43.991551 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:37:43.991609 (+ 58us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:37:43.992080 (+ 471us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:37:43.992136 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:37:43.992159 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:37:43.992304 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0807 18:37:43.992525 (+ 221us) server_negotiation.cc:299] Negotiation successful
0807 18:37:43.992669 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":339,"thread_start_us":151,"threads_started":1}
I20260807 18:37:43.994132 2641 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c6ef2450d9434c7186797dbde9cbc67c: Neither blocks nor log segments found. Creating new log.
I20260807 18:37:43.995076 2641 log.cc:824] T 00000000000000000000000000000000 P c6ef2450d9434c7186797dbde9cbc67c: Log is configured to *not* fsync() on all Append() calls
I20260807 18:37:43.996600 11321 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 18:37:43.997239 2641 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c6ef2450d9434c7186797dbde9cbc67c: No bootstrap required, opened a new log
I20260807 18:37:44.001672 2641 raft_consensus.cc:348] T 00000000000000000000000000000000 P c6ef2450d9434c7186797dbde9cbc67c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6ef2450d9434c7186797dbde9cbc67c" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 37393 } }
I20260807 18:37:44.001971 2641 raft_consensus.cc:374] T 00000000000000000000000000000000 P c6ef2450d9434c7186797dbde9cbc67c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:37:44.002041 2641 raft_consensus.cc:729] T 00000000000000000000000000000000 P c6ef2450d9434c7186797dbde9cbc67c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c6ef2450d9434c7186797dbde9cbc67c, State: Initialized, Role: FOLLOWER
I20260807 18:37:44.002590 2641 consensus_queue.cc:260] T 00000000000000000000000000000000 P c6ef2450d9434c7186797dbde9cbc67c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6ef2450d9434c7186797dbde9cbc67c" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 37393 } }
I20260807 18:37:44.002802 2641 raft_consensus.cc:388] T 00000000000000000000000000000000 P c6ef2450d9434c7186797dbde9cbc67c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:37:44.002888 2641 raft_consensus.cc:482] T 00000000000000000000000000000000 P c6ef2450d9434c7186797dbde9cbc67c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:37:44.003023 2641 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c6ef2450d9434c7186797dbde9cbc67c [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:37:44.003372 2645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:43.998727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52503 (local address 127.11.14.126:37393)
0807 18:37:43.998856 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0807 18:37:43.998860 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:37:43.999119 (+ 259us) server_negotiation.cc:407] Connection header received
0807 18:37:43.999230 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:37:43.999232 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:37:43.999277 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:37:43.999367 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:37:43.999740 (+ 373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:44.000332 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:44.001477 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:44.001727 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:44.001819 (+ 92us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:44.002827 (+ 1008us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:37:44.002830 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:37:44.002832 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:37:44.002844 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:37:44.002887 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:37:44.002904 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:37:44.003015 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0807 18:37:44.003181 (+ 166us) server_negotiation.cc:299] Negotiation successful
0807 18:37:44.003222 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260807 18:37:44.004244 2641 raft_consensus.cc:504] T 00000000000000000000000000000000 P c6ef2450d9434c7186797dbde9cbc67c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6ef2450d9434c7186797dbde9cbc67c" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 37393 } }
I20260807 18:37:44.004637 2641 leader_election.cc:302] T 00000000000000000000000000000000 P c6ef2450d9434c7186797dbde9cbc67c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c6ef2450d9434c7186797dbde9cbc67c; no voters:
I20260807 18:37:44.004974 2641 leader_election.cc:288] T 00000000000000000000000000000000 P c6ef2450d9434c7186797dbde9cbc67c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:37:44.005081 2656 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c6ef2450d9434c7186797dbde9cbc67c [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:37:44.005314 2656 raft_consensus.cc:686] T 00000000000000000000000000000000 P c6ef2450d9434c7186797dbde9cbc67c [term 1 LEADER]: Becoming Leader. State: Replica: c6ef2450d9434c7186797dbde9cbc67c, State: Running, Role: LEADER
I20260807 18:37:44.005741 2656 consensus_queue.cc:237] T 00000000000000000000000000000000 P c6ef2450d9434c7186797dbde9cbc67c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6ef2450d9434c7186797dbde9cbc67c" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 37393 } }
I20260807 18:37:44.006420 2641 sys_catalog.cc:564] T 00000000000000000000000000000000 P c6ef2450d9434c7186797dbde9cbc67c [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:37:44.007694 2658 sys_catalog.cc:455] T 00000000000000000000000000000000 P c6ef2450d9434c7186797dbde9cbc67c [sys.catalog]: SysCatalogTable state changed. Reason: New leader c6ef2450d9434c7186797dbde9cbc67c. Latest consensus state: current_term: 1 leader_uuid: "c6ef2450d9434c7186797dbde9cbc67c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6ef2450d9434c7186797dbde9cbc67c" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 37393 } } }
I20260807 18:37:44.007838 2658 sys_catalog.cc:458] T 00000000000000000000000000000000 P c6ef2450d9434c7186797dbde9cbc67c [sys.catalog]: This master's current role is: LEADER
I20260807 18:37:44.008108 2657 sys_catalog.cc:455] T 00000000000000000000000000000000 P c6ef2450d9434c7186797dbde9cbc67c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c6ef2450d9434c7186797dbde9cbc67c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6ef2450d9434c7186797dbde9cbc67c" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 37393 } } }
I20260807 18:37:44.008203 2657 sys_catalog.cc:458] T 00000000000000000000000000000000 P c6ef2450d9434c7186797dbde9cbc67c [sys.catalog]: This master's current role is: LEADER
I20260807 18:37:44.008231 2665 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:37:44.011235 2665 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:37:44.016752 2665 catalog_manager.cc:1347] Generated new cluster ID: 6175bd66ba524e3f82b6d707e27e07e0
I20260807 18:37:44.016873 2665 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:37:44.048542 2665 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:37:44.049539 2665 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:37:44.055274 2665 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c6ef2450d9434c7186797dbde9cbc67c: Generated new TSK 0
I20260807 18:37:44.056181 2665 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:37:44.068852 11321 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkz56XN/build/debug/bin/kudu with pid 2563
W20260807 18:37:44.078223 2545 connection.cc:535] client connection to 127.11.14.126:37393 recv error: Network error: failed to read from TLS socket (remote: 127.11.14.126:37393): Connection reset by peer (error 104)
2026-08-07T18:37:44Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1521 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786127579900523-11321-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:37:44 dist-test-slave-6k22 krb5kdc[2678](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:37:44 dist-test-slave-6k22 krb5kdc[2678](info): set up 2 sockets
Aug 07 18:37:44 dist-test-slave-6k22 krb5kdc[2678](info): commencing operation
krb5kdc: starting...
W20260807 18:37:45.174314 11321 mini_kdc.cc:121] Time spent starting KDC: real 1.081s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786127579900523-11321-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786127579900523-11321-0/krb5kdc/test-user.keytab.
Aug 07 18:37:45 dist-test-slave-6k22 krb5kdc[2678](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127865, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:37:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:37:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.14.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.14.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.14.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.126.keytab.
Entry for principal kudu/127.11.14.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.126.keytab.
I20260807 18:37:45.304517 11321 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkz56XN/build/debug/bin/kudu
/tmp/dist-test-taskkz56XN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786127579900523-11321-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786127579900523-11321-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786127579900523-11321-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786127579900523-11321-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.14.126:35091
--webserver_interface=127.11.14.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.126.keytab
--principal=kudu/127.11.14.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.14.84:43915
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.14.126:35091
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786127579900523-11321-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786127579900523-11321-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786127579900523-11321-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:37:45.418797 2692 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:37:45.419064 2692 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:37:45.419106 2692 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:37:45.423213 2692 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:37:45.423272 2692 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:37:45.423292 2692 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:37:45.423309 2692 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:37:45.423324 2692 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:37:45.428572 2692 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.14.84:43915
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786127579900523-11321-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786127579900523-11321-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.14.126:35091
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.126.keytab
--principal=kudu/127.11.14.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.14.126:35091
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786127579900523-11321-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.14.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2692
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786127579900523-11321-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:37:45.429623 2692 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:37:45.430804 2692 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:37:45.439541 2697 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:37:45.439689 2700 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:37:45.439622 2698 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:37:45.439646 2692 server_base.cc:1034] running on GCE node
I20260807 18:37:45.440394 2692 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:37:45.441555 2692 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:37:45.442770 2692 hybrid_clock.cc:648] HybridClock initialized: now 1786127865442746 us; error 43 us; skew 500 ppm
Aug 07 18:37:45 dist-test-slave-6k22 krb5kdc[2678](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127865, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:37:45.451202 2692 init.cc:377] Logged in from keytab as kudu/127.11.14.126@KRBTEST.COM (short username kudu)
I20260807 18:37:45.452190 2692 webserver.cc:460] Webserver started at http://127.11.14.126:41751/ using document root <none> and password file <none>
I20260807 18:37:45.452811 2692 fs_manager.cc:359] Metadata directory not provided
I20260807 18:37:45.452908 2692 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:37:45.453174 2692 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:37:45.455521 2692 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786127579900523-11321-0/minicluster-data/master-0/data/instance:
uuid: "a843ad9615ce445d9d9fdeb2f4b2c0f1"
format_stamp: "Formatted at 2026-08-07 18:37:45 on dist-test-slave-6k22"
server_key: "a52194ab97be0e31a5cd208f81d2582a"
server_key_iv: "2769e270740e8f0bd5bb4524690866e6"
server_key_version: "encryptionkey@0"
I20260807 18:37:45.456324 2692 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786127579900523-11321-0/minicluster-data/master-0/wal/instance:
uuid: "a843ad9615ce445d9d9fdeb2f4b2c0f1"
format_stamp: "Formatted at 2026-08-07 18:37:45 on dist-test-slave-6k22"
server_key: "a52194ab97be0e31a5cd208f81d2582a"
server_key_iv: "2769e270740e8f0bd5bb4524690866e6"
server_key_version: "encryptionkey@0"
I20260807 18:37:45.460228 2692 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.002s
I20260807 18:37:45.462711 2708 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:37:45.464092 2692 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 18:37:45.464242 2692 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786127579900523-11321-0/minicluster-data/master-0/data,/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786127579900523-11321-0/minicluster-data/master-0/wal
uuid: "a843ad9615ce445d9d9fdeb2f4b2c0f1"
format_stamp: "Formatted at 2026-08-07 18:37:45 on dist-test-slave-6k22"
server_key: "a52194ab97be0e31a5cd208f81d2582a"
server_key_iv: "2769e270740e8f0bd5bb4524690866e6"
server_key_version: "encryptionkey@0"
I20260807 18:37:45.464355 2692 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786127579900523-11321-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786127579900523-11321-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786127579900523-11321-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:37:45.469086 2692 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:37:45.472412 2692 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:37:45.472607 2692 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:37:45.479878 2692 rpc_server.cc:307] RPC server started. Bound to: 127.11.14.126:35091
I20260807 18:37:45.479907 2763 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.14.126:35091 every 8 connection(s)
I20260807 18:37:45.480829 2692 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786127579900523-11321-0/minicluster-data/master-0/data/info.pb
I20260807 18:37:45.481334 11321 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkz56XN/build/debug/bin/kudu as pid 2692
I20260807 18:37:45.481505 11321 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786127579900523-11321-0/minicluster-data/master-0/wal/instance
I20260807 18:37:45.481791 11321 external_mini_cluster.cc:1428] Setting key 8f0bbe81bd94241b8fe70aa5abf87200
I20260807 18:37:45.484171 2764 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:37:45.487581 2767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:45.483265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53429 (local address 127.11.14.126:35091)
0807 18:37:45.483740 (+ 475us) server_negotiation.cc:206] Beginning negotiation
0807 18:37:45.483760 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0807 18:37:45.483803 (+ 43us) server_negotiation.cc:407] Connection header received
0807 18:37:45.484653 (+ 850us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:37:45.484676 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:37:45.485035 (+ 359us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:37:45.485402 (+ 367us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:37:45.486144 (+ 742us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:37:45.486150 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:37:45.486166 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:37:45.486182 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:37:45.486248 (+ 66us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:37:45.486275 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:37:45.486414 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0807 18:37:45.486481 (+ 67us) server_negotiation.cc:299] Negotiation successful
0807 18:37:45.486627 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":319,"thread_start_us":122,"threads_started":1}
I20260807 18:37:45.490418 11321 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 18:37:45.491011 2764 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a843ad9615ce445d9d9fdeb2f4b2c0f1: Bootstrap starting.
I20260807 18:37:45.493671 2764 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a843ad9615ce445d9d9fdeb2f4b2c0f1: Neither blocks nor log segments found. Creating new log.
I20260807 18:37:45.494257 2767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:45.492512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40761 (local address 127.11.14.126:35091)
0807 18:37:45.492645 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0807 18:37:45.492649 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:37:45.492914 (+ 265us) server_negotiation.cc:407] Connection header received
0807 18:37:45.493019 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:37:45.493021 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:37:45.493088 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:37:45.493172 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:37:45.493877 (+ 705us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:37:45.493880 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:37:45.493881 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:37:45.493892 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:37:45.493916 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:37:45.493926 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:37:45.493995 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0807 18:37:45.494099 (+ 104us) server_negotiation.cc:299] Negotiation successful
0807 18:37:45.494127 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260807 18:37:45.494624 2764 log.cc:824] T 00000000000000000000000000000000 P a843ad9615ce445d9d9fdeb2f4b2c0f1: Log is configured to *not* fsync() on all Append() calls
I20260807 18:37:45.496531 2764 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a843ad9615ce445d9d9fdeb2f4b2c0f1: No bootstrap required, opened a new log
I20260807 18:37:45.499990 2764 raft_consensus.cc:348] T 00000000000000000000000000000000 P a843ad9615ce445d9d9fdeb2f4b2c0f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a843ad9615ce445d9d9fdeb2f4b2c0f1" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 35091 } }
I20260807 18:37:45.500236 2764 raft_consensus.cc:374] T 00000000000000000000000000000000 P a843ad9615ce445d9d9fdeb2f4b2c0f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:37:45.500316 2764 raft_consensus.cc:729] T 00000000000000000000000000000000 P a843ad9615ce445d9d9fdeb2f4b2c0f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a843ad9615ce445d9d9fdeb2f4b2c0f1, State: Initialized, Role: FOLLOWER
I20260807 18:37:45.500761 2764 consensus_queue.cc:260] T 00000000000000000000000000000000 P a843ad9615ce445d9d9fdeb2f4b2c0f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a843ad9615ce445d9d9fdeb2f4b2c0f1" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 35091 } }
I20260807 18:37:45.500911 2764 raft_consensus.cc:388] T 00000000000000000000000000000000 P a843ad9615ce445d9d9fdeb2f4b2c0f1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:37:45.500999 2764 raft_consensus.cc:482] T 00000000000000000000000000000000 P a843ad9615ce445d9d9fdeb2f4b2c0f1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:37:45.501097 2764 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a843ad9615ce445d9d9fdeb2f4b2c0f1 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:37:45.501919 2764 raft_consensus.cc:504] T 00000000000000000000000000000000 P a843ad9615ce445d9d9fdeb2f4b2c0f1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a843ad9615ce445d9d9fdeb2f4b2c0f1" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 35091 } }
I20260807 18:37:45.502261 2764 leader_election.cc:302] T 00000000000000000000000000000000 P a843ad9615ce445d9d9fdeb2f4b2c0f1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a843ad9615ce445d9d9fdeb2f4b2c0f1; no voters:
I20260807 18:37:45.502559 2764 leader_election.cc:288] T 00000000000000000000000000000000 P a843ad9615ce445d9d9fdeb2f4b2c0f1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:37:45.502677 2774 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a843ad9615ce445d9d9fdeb2f4b2c0f1 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:37:45.502955 2774 raft_consensus.cc:686] T 00000000000000000000000000000000 P a843ad9615ce445d9d9fdeb2f4b2c0f1 [term 1 LEADER]: Becoming Leader. State: Replica: a843ad9615ce445d9d9fdeb2f4b2c0f1, State: Running, Role: LEADER
I20260807 18:37:45.503309 2774 consensus_queue.cc:237] T 00000000000000000000000000000000 P a843ad9615ce445d9d9fdeb2f4b2c0f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a843ad9615ce445d9d9fdeb2f4b2c0f1" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 35091 } }
I20260807 18:37:45.503842 2764 sys_catalog.cc:564] T 00000000000000000000000000000000 P a843ad9615ce445d9d9fdeb2f4b2c0f1 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:37:45.505034 2775 sys_catalog.cc:455] T 00000000000000000000000000000000 P a843ad9615ce445d9d9fdeb2f4b2c0f1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a843ad9615ce445d9d9fdeb2f4b2c0f1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a843ad9615ce445d9d9fdeb2f4b2c0f1" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 35091 } } }
I20260807 18:37:45.505187 2775 sys_catalog.cc:458] T 00000000000000000000000000000000 P a843ad9615ce445d9d9fdeb2f4b2c0f1 [sys.catalog]: This master's current role is: LEADER
I20260807 18:37:45.505550 2776 sys_catalog.cc:455] T 00000000000000000000000000000000 P a843ad9615ce445d9d9fdeb2f4b2c0f1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a843ad9615ce445d9d9fdeb2f4b2c0f1. Latest consensus state: current_term: 1 leader_uuid: "a843ad9615ce445d9d9fdeb2f4b2c0f1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a843ad9615ce445d9d9fdeb2f4b2c0f1" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 35091 } } }
I20260807 18:37:45.505626 2785 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:37:45.506187 2776 sys_catalog.cc:458] T 00000000000000000000000000000000 P a843ad9615ce445d9d9fdeb2f4b2c0f1 [sys.catalog]: This master's current role is: LEADER
I20260807 18:37:45.508502 2785 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:37:45.514020 2785 catalog_manager.cc:1347] Generated new cluster ID: cbca26e353da44d6a65207be7b1e1bfd
I20260807 18:37:45.514094 2785 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:37:45.527839 2785 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:37:45.528188 2785 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:37:45.534029 2785 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a843ad9615ce445d9d9fdeb2f4b2c0f1: Generated new TSK 0
I20260807 18:37:45.534749 2785 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:37:45.545650 11321 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkz56XN/build/debug/bin/kudu with pid 2692
2026-08-07T18:37:45Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1474 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786127579900523-11321-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:37:45 dist-test-slave-6k22 krb5kdc[2800](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:37:45 dist-test-slave-6k22 krb5kdc[2800](info): set up 2 sockets
Aug 07 18:37:45 dist-test-slave-6k22 krb5kdc[2800](info): commencing operation
krb5kdc: starting...
W20260807 18:37:46.670629 11321 mini_kdc.cc:121] Time spent starting KDC: real 1.103s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786127579900523-11321-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786127579900523-11321-0/krb5kdc/test-user.keytab.
Aug 07 18:37:46 dist-test-slave-6k22 krb5kdc[2800](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127866, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:37:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:37:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.14.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.14.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.14.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.126.keytab.
Entry for principal kudu/127.11.14.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.126.keytab.
I20260807 18:37:46.804777 11321 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkz56XN/build/debug/bin/kudu
/tmp/dist-test-taskkz56XN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786127579900523-11321-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786127579900523-11321-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786127579900523-11321-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786127579900523-11321-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.14.126:42699
--webserver_interface=127.11.14.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.126.keytab
--principal=kudu/127.11.14.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.14.84:40071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.14.126:42699
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786127579900523-11321-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786127579900523-11321-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786127579900523-11321-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:37:46.926415 2814 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:37:46.926776 2814 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:37:46.926879 2814 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:37:46.931334 2814 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:37:46.931636 2814 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:37:46.931710 2814 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:37:46.931779 2814 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:37:46.931821 2814 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:37:46.937405 2814 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.14.84:40071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786127579900523-11321-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786127579900523-11321-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.14.126:42699
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.126.keytab
--principal=kudu/127.11.14.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.14.126:42699
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786127579900523-11321-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.14.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2814
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786127579900523-11321-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:37:46.938642 2814 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:37:46.939802 2814 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:37:46.948752 2821 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:37:46.948773 2823 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:37:46.948769 2820 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:37:46.949090 2814 server_base.cc:1034] running on GCE node
I20260807 18:37:46.949683 2814 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:37:46.950775 2814 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:37:46.951936 2814 hybrid_clock.cc:648] HybridClock initialized: now 1786127866951903 us; error 43 us; skew 500 ppm
Aug 07 18:37:46 dist-test-slave-6k22 krb5kdc[2800](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127866, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:37:46.954682 2814 init.cc:377] Logged in from keytab as kudu/127.11.14.126@KRBTEST.COM (short username kudu)
I20260807 18:37:46.955641 2814 webserver.cc:460] Webserver started at http://127.11.14.126:39001/ using document root <none> and password file <none>
I20260807 18:37:46.956215 2814 fs_manager.cc:359] Metadata directory not provided
I20260807 18:37:46.956331 2814 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:37:46.956574 2814 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:37:46.958659 2814 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786127579900523-11321-0/minicluster-data/master-0/data/instance:
uuid: "383c90f5634440959a6132e9f3659c5e"
format_stamp: "Formatted at 2026-08-07 18:37:46 on dist-test-slave-6k22"
server_key: "0b0430db551f7d2d0319aa621685b9eb"
server_key_iv: "77d7912cd464f472d7c7db97d45c8781"
server_key_version: "encryptionkey@0"
I20260807 18:37:46.959254 2814 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786127579900523-11321-0/minicluster-data/master-0/wal/instance:
uuid: "383c90f5634440959a6132e9f3659c5e"
format_stamp: "Formatted at 2026-08-07 18:37:46 on dist-test-slave-6k22"
server_key: "0b0430db551f7d2d0319aa621685b9eb"
server_key_iv: "77d7912cd464f472d7c7db97d45c8781"
server_key_version: "encryptionkey@0"
I20260807 18:37:46.963120 2814 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 18:37:46.966001 2830 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:37:46.967428 2814 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 18:37:46.967626 2814 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786127579900523-11321-0/minicluster-data/master-0/data,/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786127579900523-11321-0/minicluster-data/master-0/wal
uuid: "383c90f5634440959a6132e9f3659c5e"
format_stamp: "Formatted at 2026-08-07 18:37:46 on dist-test-slave-6k22"
server_key: "0b0430db551f7d2d0319aa621685b9eb"
server_key_iv: "77d7912cd464f472d7c7db97d45c8781"
server_key_version: "encryptionkey@0"
I20260807 18:37:46.967763 2814 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786127579900523-11321-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786127579900523-11321-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786127579900523-11321-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:37:46.984200 2814 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:37:46.987154 2814 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:37:46.987366 2814 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:37:46.994736 2814 rpc_server.cc:307] RPC server started. Bound to: 127.11.14.126:42699
I20260807 18:37:46.994764 2890 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.14.126:42699 every 8 connection(s)
I20260807 18:37:46.995743 2814 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786127579900523-11321-0/minicluster-data/master-0/data/info.pb
I20260807 18:37:46.998550 2892 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:37:47.001485 11321 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkz56XN/build/debug/bin/kudu as pid 2814
I20260807 18:37:47.001593 11321 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786127579900523-11321-0/minicluster-data/master-0/wal/instance
I20260807 18:37:47.001875 11321 external_mini_cluster.cc:1428] Setting key 212e1af17f355707293380483caf93c1
I20260807 18:37:47.004886 2892 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 383c90f5634440959a6132e9f3659c5e: Bootstrap starting.
I20260807 18:37:47.007659 2892 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 383c90f5634440959a6132e9f3659c5e: Neither blocks nor log segments found. Creating new log.
I20260807 18:37:47.008443 2892 log.cc:824] T 00000000000000000000000000000000 P 383c90f5634440959a6132e9f3659c5e: Log is configured to *not* fsync() on all Append() calls
I20260807 18:37:47.010365 2892 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 383c90f5634440959a6132e9f3659c5e: No bootstrap required, opened a new log
Aug 07 18:37:47 dist-test-slave-6k22 krb5kdc[2800](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127866, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.14.126@KRBTEST.COM
I20260807 18:37:47.014616 2892 raft_consensus.cc:348] T 00000000000000000000000000000000 P 383c90f5634440959a6132e9f3659c5e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "383c90f5634440959a6132e9f3659c5e" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 42699 } }
I20260807 18:37:47.014921 2892 raft_consensus.cc:374] T 00000000000000000000000000000000 P 383c90f5634440959a6132e9f3659c5e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:37:47.014992 2892 raft_consensus.cc:729] T 00000000000000000000000000000000 P 383c90f5634440959a6132e9f3659c5e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 383c90f5634440959a6132e9f3659c5e, State: Initialized, Role: FOLLOWER
I20260807 18:37:47.015473 2892 consensus_queue.cc:260] T 00000000000000000000000000000000 P 383c90f5634440959a6132e9f3659c5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "383c90f5634440959a6132e9f3659c5e" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 42699 } }
I20260807 18:37:47.015643 2892 raft_consensus.cc:388] T 00000000000000000000000000000000 P 383c90f5634440959a6132e9f3659c5e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:37:47.015736 2892 raft_consensus.cc:482] T 00000000000000000000000000000000 P 383c90f5634440959a6132e9f3659c5e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:37:47.015856 2892 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 383c90f5634440959a6132e9f3659c5e [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:37:47.017084 2892 raft_consensus.cc:504] T 00000000000000000000000000000000 P 383c90f5634440959a6132e9f3659c5e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "383c90f5634440959a6132e9f3659c5e" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 42699 } }
I20260807 18:37:47.017442 2892 leader_election.cc:302] T 00000000000000000000000000000000 P 383c90f5634440959a6132e9f3659c5e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 383c90f5634440959a6132e9f3659c5e; no voters:
I20260807 18:37:47.017779 2892 leader_election.cc:288] T 00000000000000000000000000000000 P 383c90f5634440959a6132e9f3659c5e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:37:47.017892 2897 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 383c90f5634440959a6132e9f3659c5e [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:37:47.018136 2895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:47.003330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53311 (local address 127.11.14.126:42699)
0807 18:37:47.003781 (+ 451us) server_negotiation.cc:206] Beginning negotiation
0807 18:37:47.003804 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0807 18:37:47.003853 (+ 49us) server_negotiation.cc:407] Connection header received
0807 18:37:47.004672 (+ 819us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:37:47.004696 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:37:47.005149 (+ 453us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:37:47.005521 (+ 372us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:37:47.006201 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:47.007710 (+ 1509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:47.008684 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:47.009157 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:47.009325 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:47.012138 (+ 2813us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:37:47.012145 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:37:47.012164 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:37:47.012196 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:37:47.014848 (+ 2652us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:37:47.015359 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:37:47.015364 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:37:47.015368 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:37:47.015478 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:37:47.015806 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:37:47.015810 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:37:47.015811 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:37:47.016371 (+ 560us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:37:47.016581 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0807 18:37:47.016788 (+ 207us) server_negotiation.cc:299] Negotiation successful
0807 18:37:47.017109 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":296,"thread_start_us":169,"threads_started":1}
I20260807 18:37:47.018151 2897 raft_consensus.cc:686] T 00000000000000000000000000000000 P 383c90f5634440959a6132e9f3659c5e [term 1 LEADER]: Becoming Leader. State: Replica: 383c90f5634440959a6132e9f3659c5e, State: Running, Role: LEADER
I20260807 18:37:47.018792 2897 consensus_queue.cc:237] T 00000000000000000000000000000000 P 383c90f5634440959a6132e9f3659c5e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "383c90f5634440959a6132e9f3659c5e" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 42699 } }
I20260807 18:37:47.019126 2892 sys_catalog.cc:564] T 00000000000000000000000000000000 P 383c90f5634440959a6132e9f3659c5e [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:37:47.020524 2898 sys_catalog.cc:455] T 00000000000000000000000000000000 P 383c90f5634440959a6132e9f3659c5e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 383c90f5634440959a6132e9f3659c5e. Latest consensus state: current_term: 1 leader_uuid: "383c90f5634440959a6132e9f3659c5e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "383c90f5634440959a6132e9f3659c5e" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 42699 } } }
I20260807 18:37:47.020898 2898 sys_catalog.cc:458] T 00000000000000000000000000000000 P 383c90f5634440959a6132e9f3659c5e [sys.catalog]: This master's current role is: LEADER
I20260807 18:37:47.020782 2899 sys_catalog.cc:455] T 00000000000000000000000000000000 P 383c90f5634440959a6132e9f3659c5e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "383c90f5634440959a6132e9f3659c5e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "383c90f5634440959a6132e9f3659c5e" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 42699 } } }
I20260807 18:37:47.021221 2899 sys_catalog.cc:458] T 00000000000000000000000000000000 P 383c90f5634440959a6132e9f3659c5e [sys.catalog]: This master's current role is: LEADER
I20260807 18:37:47.021767 11321 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260807 18:37:47.024562 2913 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 383c90f5634440959a6132e9f3659c5e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 18:37:47.024648 2913 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 18:37:47.024755 2912 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:37:47.025648 2912 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:37:47.031540 2912 catalog_manager.cc:1347] Generated new cluster ID: 66a7e0ecda0140d6aa4b0b6e6feec9f3
I20260807 18:37:47.031625 2912 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:37:47.033900 2895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:47.023983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48801 (local address 127.11.14.126:42699)
0807 18:37:47.024117 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0807 18:37:47.024122 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:37:47.024996 (+ 874us) server_negotiation.cc:407] Connection header received
0807 18:37:47.025109 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:37:47.025112 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:37:47.025160 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:37:47.025245 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:37:47.026088 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:47.026942 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:47.028145 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:47.028458 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:47.028565 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:47.030287 (+ 1722us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:37:47.030291 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:37:47.030293 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:37:47.030316 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:37:47.032160 (+ 1844us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:37:47.032654 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:37:47.032658 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:37:47.032660 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:37:47.032739 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:37:47.033046 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:37:47.033049 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:37:47.033051 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:37:47.033234 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:37:47.033347 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0807 18:37:47.033610 (+ 263us) server_negotiation.cc:299] Negotiation successful
0807 18:37:47.033740 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260807 18:37:47.042137 2912 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:37:47.043035 2912 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:37:47.050412 2912 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 383c90f5634440959a6132e9f3659c5e: Generated new TSK 0
I20260807 18:37:47.051049 2912 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:37:47.057442 11321 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkz56XN/build/debug/bin/kudu with pid 2814
2026-08-07T18:37:47Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1514 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786127579900523-11321-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:37:47 dist-test-slave-6k22 krb5kdc[2925](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:37:47 dist-test-slave-6k22 krb5kdc[2925](info): set up 2 sockets
Aug 07 18:37:47 dist-test-slave-6k22 krb5kdc[2925](info): commencing operation
krb5kdc: starting...
W20260807 18:37:48.162305 11321 mini_kdc.cc:121] Time spent starting KDC: real 1.080s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786127579900523-11321-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786127579900523-11321-0/krb5kdc/test-user.keytab.
Aug 07 18:37:48 dist-test-slave-6k22 krb5kdc[2925](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127868, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:37:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:37:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.14.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.14.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.14.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.126.keytab.
Entry for principal kudu/127.11.14.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.126.keytab.
I20260807 18:37:48.299129 11321 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkz56XN/build/debug/bin/kudu
/tmp/dist-test-taskkz56XN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786127579900523-11321-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786127579900523-11321-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786127579900523-11321-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786127579900523-11321-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.14.126:42837
--webserver_interface=127.11.14.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.126.keytab
--principal=kudu/127.11.14.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.14.84:41383
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.14.126:42837
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786127579900523-11321-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786127579900523-11321-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786127579900523-11321-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:37:48.412792 2939 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:37:48.413058 2939 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:37:48.413103 2939 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:37:48.417083 2939 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:37:48.417137 2939 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:37:48.417156 2939 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:37:48.417173 2939 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:37:48.417188 2939 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:37:48.422334 2939 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.14.84:41383
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786127579900523-11321-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786127579900523-11321-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.14.126:42837
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.126.keytab
--principal=kudu/127.11.14.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.14.126:42837
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786127579900523-11321-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.14.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2939
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786127579900523-11321-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:37:48.423382 2939 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:37:48.424481 2939 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:37:48.433012 2949 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:37:48.433261 2939 server_base.cc:1034] running on GCE node
W20260807 18:37:48.433022 2947 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:37:48.433038 2946 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:37:48.434046 2939 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:37:48.435168 2939 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:37:48.436334 2939 hybrid_clock.cc:648] HybridClock initialized: now 1786127868436320 us; error 34 us; skew 500 ppm
Aug 07 18:37:48 dist-test-slave-6k22 krb5kdc[2925](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127868, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:37:48.439289 2939 init.cc:377] Logged in from keytab as kudu/127.11.14.126@KRBTEST.COM (short username kudu)
I20260807 18:37:48.440311 2939 webserver.cc:460] Webserver started at http://127.11.14.126:38715/ using document root <none> and password file <none>
I20260807 18:37:48.440876 2939 fs_manager.cc:359] Metadata directory not provided
I20260807 18:37:48.440940 2939 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:37:48.441123 2939 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:37:48.443341 2939 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786127579900523-11321-0/minicluster-data/master-0/data/instance:
uuid: "ef227d871dbe420eb507e1c4b02a48a3"
format_stamp: "Formatted at 2026-08-07 18:37:48 on dist-test-slave-6k22"
server_key: "649c27733bc2c30e305206e5fb381f46"
server_key_iv: "509b5960264562439349a1fd7ac55dfb"
server_key_version: "encryptionkey@0"
I20260807 18:37:48.443992 2939 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786127579900523-11321-0/minicluster-data/master-0/wal/instance:
uuid: "ef227d871dbe420eb507e1c4b02a48a3"
format_stamp: "Formatted at 2026-08-07 18:37:48 on dist-test-slave-6k22"
server_key: "649c27733bc2c30e305206e5fb381f46"
server_key_iv: "509b5960264562439349a1fd7ac55dfb"
server_key_version: "encryptionkey@0"
I20260807 18:37:48.448045 2939 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 18:37:48.450680 2958 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:37:48.452164 2939 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:37:48.452314 2939 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786127579900523-11321-0/minicluster-data/master-0/data,/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786127579900523-11321-0/minicluster-data/master-0/wal
uuid: "ef227d871dbe420eb507e1c4b02a48a3"
format_stamp: "Formatted at 2026-08-07 18:37:48 on dist-test-slave-6k22"
server_key: "649c27733bc2c30e305206e5fb381f46"
server_key_iv: "509b5960264562439349a1fd7ac55dfb"
server_key_version: "encryptionkey@0"
I20260807 18:37:48.452425 2939 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786127579900523-11321-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786127579900523-11321-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786127579900523-11321-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:37:48.487334 2939 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:37:48.491175 2939 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:37:48.491353 2939 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:37:48.499715 2939 rpc_server.cc:307] RPC server started. Bound to: 127.11.14.126:42837
I20260807 18:37:48.499749 3016 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.14.126:42837 every 8 connection(s)
I20260807 18:37:48.500871 2939 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786127579900523-11321-0/minicluster-data/master-0/data/info.pb
I20260807 18:37:48.503799 3017 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:37:48.506712 11321 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkz56XN/build/debug/bin/kudu as pid 2939
I20260807 18:37:48.506879 11321 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786127579900523-11321-0/minicluster-data/master-0/wal/instance
I20260807 18:37:48.507150 11321 external_mini_cluster.cc:1428] Setting key 4eb60d5911e8e9241a782ccfd112356c
I20260807 18:37:48.510085 3017 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ef227d871dbe420eb507e1c4b02a48a3: Bootstrap starting.
I20260807 18:37:48.512723 3017 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ef227d871dbe420eb507e1c4b02a48a3: Neither blocks nor log segments found. Creating new log.
I20260807 18:37:48.513737 3017 log.cc:824] T 00000000000000000000000000000000 P ef227d871dbe420eb507e1c4b02a48a3: Log is configured to *not* fsync() on all Append() calls
I20260807 18:37:48.516489 3017 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ef227d871dbe420eb507e1c4b02a48a3: No bootstrap required, opened a new log
I20260807 18:37:48.517728 3020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:48.508595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36443 (local address 127.11.14.126:42837)
0807 18:37:48.509043 (+ 448us) server_negotiation.cc:206] Beginning negotiation
0807 18:37:48.509061 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0807 18:37:48.509127 (+ 66us) server_negotiation.cc:407] Connection header received
0807 18:37:48.509860 (+ 733us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:37:48.509880 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:37:48.510299 (+ 419us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:37:48.510618 (+ 319us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:37:48.510985 (+ 367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:48.512161 (+ 1176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:48.513447 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:48.514465 (+ 1018us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:48.514608 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:48.515524 (+ 916us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:37:48.515533 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:37:48.515550 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:37:48.516141 (+ 591us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:37:48.516220 (+ 79us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:37:48.516257 (+ 37us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:37:48.516373 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0807 18:37:48.516480 (+ 107us) server_negotiation.cc:299] Negotiation successful
0807 18:37:48.516723 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":285,"thread_start_us":153,"threads_started":1}
I20260807 18:37:48.520869 11321 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 18:37:48.521350 3017 raft_consensus.cc:348] T 00000000000000000000000000000000 P ef227d871dbe420eb507e1c4b02a48a3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef227d871dbe420eb507e1c4b02a48a3" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 42837 } }
I20260807 18:37:48.521581 3017 raft_consensus.cc:374] T 00000000000000000000000000000000 P ef227d871dbe420eb507e1c4b02a48a3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:37:48.521670 3017 raft_consensus.cc:729] T 00000000000000000000000000000000 P ef227d871dbe420eb507e1c4b02a48a3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef227d871dbe420eb507e1c4b02a48a3, State: Initialized, Role: FOLLOWER
I20260807 18:37:48.522123 3017 consensus_queue.cc:260] T 00000000000000000000000000000000 P ef227d871dbe420eb507e1c4b02a48a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef227d871dbe420eb507e1c4b02a48a3" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 42837 } }
I20260807 18:37:48.522343 3017 raft_consensus.cc:388] T 00000000000000000000000000000000 P ef227d871dbe420eb507e1c4b02a48a3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:37:48.522429 3017 raft_consensus.cc:482] T 00000000000000000000000000000000 P ef227d871dbe420eb507e1c4b02a48a3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:37:48.522542 3017 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ef227d871dbe420eb507e1c4b02a48a3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:37:48.523967 3017 raft_consensus.cc:504] T 00000000000000000000000000000000 P ef227d871dbe420eb507e1c4b02a48a3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef227d871dbe420eb507e1c4b02a48a3" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 42837 } }
I20260807 18:37:48.524369 3017 leader_election.cc:302] T 00000000000000000000000000000000 P ef227d871dbe420eb507e1c4b02a48a3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ef227d871dbe420eb507e1c4b02a48a3; no voters:
I20260807 18:37:48.524703 3017 leader_election.cc:288] T 00000000000000000000000000000000 P ef227d871dbe420eb507e1c4b02a48a3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:37:48.524811 3027 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ef227d871dbe420eb507e1c4b02a48a3 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:37:48.525044 3027 raft_consensus.cc:686] T 00000000000000000000000000000000 P ef227d871dbe420eb507e1c4b02a48a3 [term 1 LEADER]: Becoming Leader. State: Replica: ef227d871dbe420eb507e1c4b02a48a3, State: Running, Role: LEADER
I20260807 18:37:48.525431 3027 consensus_queue.cc:237] T 00000000000000000000000000000000 P ef227d871dbe420eb507e1c4b02a48a3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef227d871dbe420eb507e1c4b02a48a3" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 42837 } }
I20260807 18:37:48.526084 3017 sys_catalog.cc:564] T 00000000000000000000000000000000 P ef227d871dbe420eb507e1c4b02a48a3 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:37:48.527604 3028 sys_catalog.cc:455] T 00000000000000000000000000000000 P ef227d871dbe420eb507e1c4b02a48a3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ef227d871dbe420eb507e1c4b02a48a3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef227d871dbe420eb507e1c4b02a48a3" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 42837 } } }
I20260807 18:37:48.527765 3028 sys_catalog.cc:458] T 00000000000000000000000000000000 P ef227d871dbe420eb507e1c4b02a48a3 [sys.catalog]: This master's current role is: LEADER
I20260807 18:37:48.528016 3020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:48.523039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51975 (local address 127.11.14.126:42837)
0807 18:37:48.523160 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0807 18:37:48.523164 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:37:48.523400 (+ 236us) server_negotiation.cc:407] Connection header received
0807 18:37:48.523469 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:37:48.523472 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:37:48.523523 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:37:48.523651 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:37:48.524123 (+ 472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:48.524718 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:48.525848 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:48.526269 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:48.526379 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:48.527305 (+ 926us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:37:48.527309 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:37:48.527313 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:37:48.527328 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:37:48.527356 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:37:48.527372 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:37:48.527444 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0807 18:37:48.527807 (+ 363us) server_negotiation.cc:299] Negotiation successful
0807 18:37:48.527879 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260807 18:37:48.528362 3029 sys_catalog.cc:455] T 00000000000000000000000000000000 P ef227d871dbe420eb507e1c4b02a48a3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ef227d871dbe420eb507e1c4b02a48a3. Latest consensus state: current_term: 1 leader_uuid: "ef227d871dbe420eb507e1c4b02a48a3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef227d871dbe420eb507e1c4b02a48a3" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 42837 } } }
I20260807 18:37:48.528541 3029 sys_catalog.cc:458] T 00000000000000000000000000000000 P ef227d871dbe420eb507e1c4b02a48a3 [sys.catalog]: This master's current role is: LEADER
W20260807 18:37:48.531958 3045 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ef227d871dbe420eb507e1c4b02a48a3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 18:37:48.532059 3045 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 18:37:48.532135 3037 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:37:48.532907 3037 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:37:48.538699 3037 catalog_manager.cc:1347] Generated new cluster ID: b0256e75f6c84e839f9e58728b3750fe
I20260807 18:37:48.538767 3037 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:37:48.552772 3037 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:37:48.553767 3037 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:37:48.569952 3037 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ef227d871dbe420eb507e1c4b02a48a3: Generated new TSK 0
I20260807 18:37:48.570730 3037 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:37:48.581660 11321 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkz56XN/build/debug/bin/kudu with pid 2939
2026-08-07T18:37:48Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1523 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786127579900523-11321-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:37:48 dist-test-slave-6k22 krb5kdc[3050](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:37:48 dist-test-slave-6k22 krb5kdc[3050](info): set up 2 sockets
Aug 07 18:37:48 dist-test-slave-6k22 krb5kdc[3050](info): commencing operation
krb5kdc: starting...
W20260807 18:37:49.670347 11321 mini_kdc.cc:121] Time spent starting KDC: real 1.065s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786127579900523-11321-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786127579900523-11321-0/krb5kdc/test-user.keytab.
Aug 07 18:37:49 dist-test-slave-6k22 krb5kdc[3050](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127869, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:37:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:37:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.14.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.14.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.14.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.126.keytab.
Entry for principal kudu/127.11.14.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.126.keytab.
I20260807 18:37:49.810623 11321 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkz56XN/build/debug/bin/kudu
/tmp/dist-test-taskkz56XN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786127579900523-11321-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786127579900523-11321-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786127579900523-11321-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786127579900523-11321-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.14.126:41523
--webserver_interface=127.11.14.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.126.keytab
--principal=kudu/127.11.14.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.14.84:34535
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.14.126:41523
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786127579900523-11321-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786127579900523-11321-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786127579900523-11321-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:37:49.928851 3064 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:37:49.929143 3064 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:37:49.929191 3064 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:37:49.933422 3064 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:37:49.933501 3064 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:37:49.933522 3064 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:37:49.933539 3064 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:37:49.933555 3064 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:37:49.939141 3064 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.14.84:34535
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786127579900523-11321-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786127579900523-11321-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.14.126:41523
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.126.keytab
--principal=kudu/127.11.14.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.14.126:41523
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786127579900523-11321-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.14.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3064
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786127579900523-11321-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:37:49.940569 3064 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:37:49.941808 3064 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:37:49.950874 3070 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:37:49.950867 3072 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:37:49.951018 3069 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:37:49.951318 3064 server_base.cc:1034] running on GCE node
I20260807 18:37:49.951925 3064 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:37:49.953075 3064 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:37:49.954267 3064 hybrid_clock.cc:648] HybridClock initialized: now 1786127869954240 us; error 40 us; skew 500 ppm
Aug 07 18:37:49 dist-test-slave-6k22 krb5kdc[3050](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127869, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:37:49.957268 3064 init.cc:377] Logged in from keytab as kudu/127.11.14.126@KRBTEST.COM (short username kudu)
I20260807 18:37:49.958249 3064 webserver.cc:460] Webserver started at http://127.11.14.126:37437/ using document root <none> and password file <none>
I20260807 18:37:49.958844 3064 fs_manager.cc:359] Metadata directory not provided
I20260807 18:37:49.958940 3064 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:37:49.959193 3064 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:37:49.961298 3064 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786127579900523-11321-0/minicluster-data/master-0/data/instance:
uuid: "31d696f80c6044c4a062e5feb2eb773e"
format_stamp: "Formatted at 2026-08-07 18:37:49 on dist-test-slave-6k22"
server_key: "8dcba28457e286aaacd4e33b4b5fdd54"
server_key_iv: "5ec28803c3f6af3d7f4a2f247041c98c"
server_key_version: "encryptionkey@0"
I20260807 18:37:49.961877 3064 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786127579900523-11321-0/minicluster-data/master-0/wal/instance:
uuid: "31d696f80c6044c4a062e5feb2eb773e"
format_stamp: "Formatted at 2026-08-07 18:37:49 on dist-test-slave-6k22"
server_key: "8dcba28457e286aaacd4e33b4b5fdd54"
server_key_iv: "5ec28803c3f6af3d7f4a2f247041c98c"
server_key_version: "encryptionkey@0"
I20260807 18:37:49.965721 3064 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.002s
I20260807 18:37:49.968175 3080 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:37:49.969497 3064 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 18:37:49.969642 3064 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786127579900523-11321-0/minicluster-data/master-0/data,/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786127579900523-11321-0/minicluster-data/master-0/wal
uuid: "31d696f80c6044c4a062e5feb2eb773e"
format_stamp: "Formatted at 2026-08-07 18:37:49 on dist-test-slave-6k22"
server_key: "8dcba28457e286aaacd4e33b4b5fdd54"
server_key_iv: "5ec28803c3f6af3d7f4a2f247041c98c"
server_key_version: "encryptionkey@0"
I20260807 18:37:49.969756 3064 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786127579900523-11321-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786127579900523-11321-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786127579900523-11321-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:37:49.974473 3064 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:37:49.977392 3064 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:37:49.977591 3064 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:37:49.984766 3064 rpc_server.cc:307] RPC server started. Bound to: 127.11.14.126:41523
I20260807 18:37:49.984795 3141 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.14.126:41523 every 8 connection(s)
I20260807 18:37:49.985709 3064 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786127579900523-11321-0/minicluster-data/master-0/data/info.pb
I20260807 18:37:49.987407 11321 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkz56XN/build/debug/bin/kudu as pid 3064
I20260807 18:37:49.987586 11321 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786127579900523-11321-0/minicluster-data/master-0/wal/instance
I20260807 18:37:49.987898 11321 external_mini_cluster.cc:1428] Setting key a7e188ae7dc8ac8086fec9116175f77e
I20260807 18:37:49.988929 3143 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:37:49.993227 3147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:49.989329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52601 (local address 127.11.14.126:41523)
0807 18:37:49.989843 (+ 514us) server_negotiation.cc:206] Beginning negotiation
0807 18:37:49.989858 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 18:37:49.989896 (+ 38us) server_negotiation.cc:407] Connection header received
0807 18:37:49.990554 (+ 658us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:37:49.990572 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:37:49.990864 (+ 292us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:37:49.991201 (+ 337us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:37:49.992051 (+ 850us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:37:49.992057 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:37:49.992080 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:37:49.992090 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:37:49.992143 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:37:49.992161 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:37:49.992314 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0807 18:37:49.992387 (+ 73us) server_negotiation.cc:299] Negotiation successful
0807 18:37:49.992538 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":370,"thread_start_us":168,"threads_started":1}
I20260807 18:37:49.995298 3143 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 31d696f80c6044c4a062e5feb2eb773e: Bootstrap starting.
I20260807 18:37:49.996343 11321 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 18:37:49.997804 3143 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 31d696f80c6044c4a062e5feb2eb773e: Neither blocks nor log segments found. Creating new log.
I20260807 18:37:49.998638 3143 log.cc:824] T 00000000000000000000000000000000 P 31d696f80c6044c4a062e5feb2eb773e: Log is configured to *not* fsync() on all Append() calls
I20260807 18:37:50.000283 3147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:49.998530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42305 (local address 127.11.14.126:41523)
0807 18:37:49.998663 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0807 18:37:49.998669 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:37:49.998950 (+ 281us) server_negotiation.cc:407] Connection header received
0807 18:37:49.999030 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:37:49.999033 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:37:49.999069 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:37:49.999146 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:37:49.999937 (+ 791us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:37:49.999940 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:37:49.999943 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:37:49.999955 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:37:49.999984 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:37:49.999996 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:37:50.000089 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0807 18:37:50.000125 (+ 36us) server_negotiation.cc:299] Negotiation successful
0807 18:37:50.000160 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260807 18:37:50.000963 3143 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 31d696f80c6044c4a062e5feb2eb773e: No bootstrap required, opened a new log
I20260807 18:37:50.004690 3143 raft_consensus.cc:348] T 00000000000000000000000000000000 P 31d696f80c6044c4a062e5feb2eb773e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31d696f80c6044c4a062e5feb2eb773e" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 41523 } }
I20260807 18:37:50.004912 3143 raft_consensus.cc:374] T 00000000000000000000000000000000 P 31d696f80c6044c4a062e5feb2eb773e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:37:50.004976 3143 raft_consensus.cc:729] T 00000000000000000000000000000000 P 31d696f80c6044c4a062e5feb2eb773e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31d696f80c6044c4a062e5feb2eb773e, State: Initialized, Role: FOLLOWER
I20260807 18:37:50.005457 3143 consensus_queue.cc:260] T 00000000000000000000000000000000 P 31d696f80c6044c4a062e5feb2eb773e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31d696f80c6044c4a062e5feb2eb773e" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 41523 } }
I20260807 18:37:50.005676 3156 raft_consensus.cc:482] T 00000000000000000000000000000000 P 31d696f80c6044c4a062e5feb2eb773e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:37:50.005885 3156 raft_consensus.cc:504] T 00000000000000000000000000000000 P 31d696f80c6044c4a062e5feb2eb773e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31d696f80c6044c4a062e5feb2eb773e" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 41523 } }
I20260807 18:37:50.005620 3143 raft_consensus.cc:388] T 00000000000000000000000000000000 P 31d696f80c6044c4a062e5feb2eb773e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:37:50.006366 3143 raft_consensus.cc:482] T 00000000000000000000000000000000 P 31d696f80c6044c4a062e5feb2eb773e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:37:50.006315 3156 leader_election.cc:302] T 00000000000000000000000000000000 P 31d696f80c6044c4a062e5feb2eb773e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 31d696f80c6044c4a062e5feb2eb773e; no voters:
I20260807 18:37:50.006450 3143 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 31d696f80c6044c4a062e5feb2eb773e [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:37:50.006742 3156 leader_election.cc:288] T 00000000000000000000000000000000 P 31d696f80c6044c4a062e5feb2eb773e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers
I20260807 18:37:50.007547 3143 raft_consensus.cc:504] T 00000000000000000000000000000000 P 31d696f80c6044c4a062e5feb2eb773e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31d696f80c6044c4a062e5feb2eb773e" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 41523 } }
I20260807 18:37:50.007750 3143 leader_election.cc:302] T 00000000000000000000000000000000 P 31d696f80c6044c4a062e5feb2eb773e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 31d696f80c6044c4a062e5feb2eb773e; no voters:
I20260807 18:37:50.007848 3157 raft_consensus.cc:2741] T 00000000000000000000000000000000 P 31d696f80c6044c4a062e5feb2eb773e [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260807 18:37:50.007893 3143 leader_election.cc:288] T 00000000000000000000000000000000 P 31d696f80c6044c4a062e5feb2eb773e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:37:50.008108 3156 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 31d696f80c6044c4a062e5feb2eb773e [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:37:50.008391 3156 raft_consensus.cc:686] T 00000000000000000000000000000000 P 31d696f80c6044c4a062e5feb2eb773e [term 1 LEADER]: Becoming Leader. State: Replica: 31d696f80c6044c4a062e5feb2eb773e, State: Running, Role: LEADER
I20260807 18:37:50.008764 3156 consensus_queue.cc:237] T 00000000000000000000000000000000 P 31d696f80c6044c4a062e5feb2eb773e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31d696f80c6044c4a062e5feb2eb773e" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 41523 } }
I20260807 18:37:50.009244 3143 sys_catalog.cc:564] T 00000000000000000000000000000000 P 31d696f80c6044c4a062e5feb2eb773e [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:37:50.010288 3158 sys_catalog.cc:455] T 00000000000000000000000000000000 P 31d696f80c6044c4a062e5feb2eb773e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "31d696f80c6044c4a062e5feb2eb773e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31d696f80c6044c4a062e5feb2eb773e" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 41523 } } }
I20260807 18:37:50.010335 3159 sys_catalog.cc:455] T 00000000000000000000000000000000 P 31d696f80c6044c4a062e5feb2eb773e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 31d696f80c6044c4a062e5feb2eb773e. Latest consensus state: current_term: 1 leader_uuid: "31d696f80c6044c4a062e5feb2eb773e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31d696f80c6044c4a062e5feb2eb773e" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 41523 } } }
I20260807 18:37:50.010473 3158 sys_catalog.cc:458] T 00000000000000000000000000000000 P 31d696f80c6044c4a062e5feb2eb773e [sys.catalog]: This master's current role is: LEADER
I20260807 18:37:50.010473 3159 sys_catalog.cc:458] T 00000000000000000000000000000000 P 31d696f80c6044c4a062e5feb2eb773e [sys.catalog]: This master's current role is: LEADER
I20260807 18:37:50.010972 3167 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:37:50.014438 3167 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:37:50.020716 3167 catalog_manager.cc:1347] Generated new cluster ID: 56d3fab490e04f45a7c63752493689ad
I20260807 18:37:50.020798 3167 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:37:50.043215 3167 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:37:50.043582 3167 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:37:50.050031 3167 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 31d696f80c6044c4a062e5feb2eb773e: Generated new TSK 0
I20260807 18:37:50.050704 3167 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:37:50.054607 11321 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkz56XN/build/debug/bin/kudu with pid 3064
2026-08-07T18:37:50Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1474 ms)
[----------] 18 tests from AuthTokenIssuingTest (27630 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786127579900523-11321-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:37:50 dist-test-slave-6k22 krb5kdc[3179](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:37:50 dist-test-slave-6k22 krb5kdc[3179](info): set up 2 sockets
Aug 07 18:37:50 dist-test-slave-6k22 krb5kdc[3179](info): commencing operation
krb5kdc: starting...
W20260807 18:37:51.234592 11321 mini_kdc.cc:121] Time spent starting KDC: real 1.155s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786127579900523-11321-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786127579900523-11321-0/krb5kdc/test-user.keytab.
Aug 07 18:37:51 dist-test-slave-6k22 krb5kdc[3179](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127871, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:37:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:37:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.14.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.14.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.14.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.126.keytab.
Entry for principal kudu/127.11.14.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.126.keytab.
I20260807 18:37:51.362687 11321 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkz56XN/build/debug/bin/kudu
/tmp/dist-test-taskkz56XN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786127579900523-11321-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786127579900523-11321-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786127579900523-11321-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786127579900523-11321-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.14.126:45853
--webserver_interface=127.11.14.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.126.keytab
--principal=kudu/127.11.14.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.14.84:40635
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.14.126:45853,127.11.14.125:37885,127.11.14.124:35709
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786127579900523-11321-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786127579900523-11321-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786127579900523-11321-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:37:51.479102 3194 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 18:37:51.479485 3194 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:37:51.479566 3194 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:37:51.479683 3194 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:37:51.483831 3194 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:37:51.483898 3194 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:37:51.483918 3194 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:37:51.483937 3194 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:37:51.483951 3194 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:37:51.489336 3194 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.14.84:40635
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786127579900523-11321-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786127579900523-11321-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.14.126:45853,127.11.14.125:37885,127.11.14.124:35709
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.126.keytab
--principal=kudu/127.11.14.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.14.126:45853
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786127579900523-11321-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.14.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3194
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786127579900523-11321-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:37:51.490494 3194 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:37:51.491581 3194 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:37:51.500188 3202 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:37:51.500165 3200 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:37:51.500165 3199 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:37:51.500577 3194 server_base.cc:1034] running on GCE node
I20260807 18:37:51.501222 3194 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:37:51.502406 3194 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:37:51.503628 3194 hybrid_clock.cc:648] HybridClock initialized: now 1786127871503600 us; error 50 us; skew 500 ppm
Aug 07 18:37:51 dist-test-slave-6k22 krb5kdc[3179](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127871, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:37:51.513615 3194 init.cc:377] Logged in from keytab as kudu/127.11.14.126@KRBTEST.COM (short username kudu)
I20260807 18:37:51.514654 3194 webserver.cc:460] Webserver started at http://127.11.14.126:33053/ using document root <none> and password file <none>
I20260807 18:37:51.515293 3194 fs_manager.cc:359] Metadata directory not provided
I20260807 18:37:51.515398 3194 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:37:51.515724 3194 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:37:51.517915 3194 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786127579900523-11321-0/minicluster-data/master-0/data/instance:
uuid: "ac07d9da9d3f460caecd1af62b86a26f"
format_stamp: "Formatted at 2026-08-07 18:37:51 on dist-test-slave-6k22"
server_key: "eca9a69ed9d57d6d65054cb20e8ac6d6"
server_key_iv: "b3a9bbcf53115321c05e630a1292375b"
server_key_version: "encryptionkey@0"
I20260807 18:37:51.518555 3194 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786127579900523-11321-0/minicluster-data/master-0/wal/instance:
uuid: "ac07d9da9d3f460caecd1af62b86a26f"
format_stamp: "Formatted at 2026-08-07 18:37:51 on dist-test-slave-6k22"
server_key: "eca9a69ed9d57d6d65054cb20e8ac6d6"
server_key_iv: "b3a9bbcf53115321c05e630a1292375b"
server_key_version: "encryptionkey@0"
I20260807 18:37:51.522765 3194 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.002s
I20260807 18:37:51.525300 3209 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:37:51.526546 3194 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 18:37:51.526700 3194 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786127579900523-11321-0/minicluster-data/master-0/data,/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786127579900523-11321-0/minicluster-data/master-0/wal
uuid: "ac07d9da9d3f460caecd1af62b86a26f"
format_stamp: "Formatted at 2026-08-07 18:37:51 on dist-test-slave-6k22"
server_key: "eca9a69ed9d57d6d65054cb20e8ac6d6"
server_key_iv: "b3a9bbcf53115321c05e630a1292375b"
server_key_version: "encryptionkey@0"
I20260807 18:37:51.526834 3194 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786127579900523-11321-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786127579900523-11321-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786127579900523-11321-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:37:51.544616 3194 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:37:51.547544 3194 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:37:51.547793 3194 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:37:51.554948 3194 rpc_server.cc:307] RPC server started. Bound to: 127.11.14.126:45853
I20260807 18:37:51.555218 3268 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.14.126:45853 every 8 connection(s)
I20260807 18:37:51.555984 3194 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786127579900523-11321-0/minicluster-data/master-0/data/info.pb
I20260807 18:37:51.559065 3269 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:37:51.559801 11321 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkz56XN/build/debug/bin/kudu as pid 3194
I20260807 18:37:51.559940 11321 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786127579900523-11321-0/minicluster-data/master-0/wal/instance
I20260807 18:37:51.560204 11321 external_mini_cluster.cc:1428] Setting key c6838cb4f3ff57474f2f669824a0ecfc
I20260807 18:37:51.563068 3269 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.14.126" port: 45853 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.11.14.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.14.125@KRBTEST.COM" created.
Aug 07 18:37:51 dist-test-slave-6k22 krb5kdc[3179](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127871, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.126@KRBTEST.COM for kudu/127.11.14.126@KRBTEST.COM
I20260807 18:37:51.578773 3274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:51.565491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:45853 (local address 127.0.0.1:49680)
0807 18:37:51.566055 (+ 564us) negotiation.cc:107] Waiting for socket to connect
0807 18:37:51.566090 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0807 18:37:51.566950 (+ 860us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:37:51.567681 (+ 731us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:37:51.567687 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:37:51.568148 (+ 461us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:37:51.568402 (+ 254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:37:51.568416 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:37:51.569612 (+ 1196us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:37:51.569617 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:37:51.570341 (+ 724us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:37:51.570349 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:37:51.570986 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:37:51.570990 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:37:51.571097 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:51.571761 (+ 664us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:37:51.571778 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:37:51.573266 (+ 1488us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:37:51.575582 (+ 2316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:37:51.575591 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:37:51.575601 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:37:51.575834 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:37:51.576164 (+ 330us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:37:51.576167 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:37:51.576169 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:37:51.576283 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:37:51.576969 (+ 686us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:37:51.576976 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:37:51.577131 (+ 155us) client_negotiation.cc:769] Sending connection context
0807 18:37:51.577307 (+ 176us) client_negotiation.cc:240] Negotiation successful
0807 18:37:51.577731 (+ 424us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":403,"mutex_wait_us":141,"thread_start_us":163,"threads_started":1}
I20260807 18:37:51.578773 3273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:51.565489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49680 (local address 127.11.14.126:45853)
0807 18:37:51.565902 (+ 413us) server_negotiation.cc:206] Beginning negotiation
0807 18:37:51.565922 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0807 18:37:51.566199 (+ 277us) server_negotiation.cc:407] Connection header received
0807 18:37:51.567269 (+ 1070us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:37:51.567274 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:37:51.567458 (+ 184us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:37:51.567690 (+ 232us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:37:51.568551 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:51.569487 (+ 936us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:51.570458 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:51.570820 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:51.570986 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:51.573466 (+ 2480us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:37:51.573471 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:37:51.573482 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:37:51.573501 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:37:51.575455 (+ 1954us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:37:51.575948 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:37:51.575954 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:37:51.575955 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:37:51.576064 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:37:51.576382 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:37:51.576387 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:37:51.576388 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:37:51.576850 (+ 462us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:37:51.576989 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0807 18:37:51.577375 (+ 386us) server_negotiation.cc:299] Negotiation successful
0807 18:37:51.577680 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":18,"server-negotiator.queue_time_us":265,"thread_start_us":129,"threads_started":1}
I20260807 18:37:51.581617 3269 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.14.125" port: 37885 } has no permanent_uuid. Determining permanent_uuid...
I20260807 18:37:51.582450 3274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:51.582158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.125:37885 (local address 127.0.0.1:42754)
0807 18:37:51.582281 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0807 18:37:51.582386 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.125:37885: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260807 18:37:51.582641 3212 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.11.14.125:37885: connect: Connection refused (error 111)
I20260807 18:37:51.583531 3274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:51.583307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.125:37885 (local address 127.0.0.1:42762)
0807 18:37:51.583435 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0807 18:37:51.583479 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.125:37885: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260807 18:37:51.583690 3269 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.14.125:37885: Network error: Client connection negotiation failed: client connection to 127.11.14.125:37885: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.14.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.125.keytab.
Entry for principal kudu/127.11.14.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.125.keytab.
I20260807 18:37:51.590732 11321 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkz56XN/build/debug/bin/kudu
/tmp/dist-test-taskkz56XN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786127579900523-11321-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786127579900523-11321-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786127579900523-11321-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786127579900523-11321-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.14.125:37885
--webserver_interface=127.11.14.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.125.keytab
--principal=kudu/127.11.14.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.14.84:40635
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.14.126:45853,127.11.14.125:37885,127.11.14.124:35709
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786127579900523-11321-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786127579900523-11321-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786127579900523-11321-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 18:37:51.633080 3269 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.14.125" port: 37885 } attempt: 1
I20260807 18:37:51.634083 3274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:51.633674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.125:37885 (local address 127.0.0.1:42776)
0807 18:37:51.633876 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0807 18:37:51.633967 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.125:37885: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260807 18:37:51.634965 3274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:51.634670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.125:37885 (local address 127.0.0.1:42782)
0807 18:37:51.634805 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0807 18:37:51.634874 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.125:37885: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260807 18:37:51.635149 3269 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.14.125:37885: Network error: Client connection negotiation failed: client connection to 127.11.14.125:37885: connect: Connection refused (error 111)
I20260807 18:37:51.703336 3269 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.14.125" port: 37885 } attempt: 2
I20260807 18:37:51.704264 3274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:51.703910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.125:37885 (local address 127.0.0.1:42794)
0807 18:37:51.704086 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0807 18:37:51.704159 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.125:37885: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260807 18:37:51.705070 3274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:51.704851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.125:37885 (local address 127.0.0.1:42802)
0807 18:37:51.704956 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0807 18:37:51.705008 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.125:37885: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260807 18:37:51.705312 3269 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.14.125:37885: Network error: Client connection negotiation failed: client connection to 127.11.14.125:37885: connect: Connection refused (error 111)
W20260807 18:37:51.706425 3277 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 18:37:51.706753 3277 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:37:51.706807 3277 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:37:51.706926 3277 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:37:51.710906 3277 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:37:51.710980 3277 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:37:51.711026 3277 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:37:51.711081 3277 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:37:51.711118 3277 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:37:51.716320 3277 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.14.84:40635
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786127579900523-11321-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786127579900523-11321-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.11.14.126:45853,127.11.14.125:37885,127.11.14.124:35709
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.125.keytab
--principal=kudu/127.11.14.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.14.125:37885
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786127579900523-11321-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.11.14.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3277
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786127579900523-11321-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:37:51.717403 3277 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:37:51.718425 3277 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:37:51.726877 3284 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:37:51.726847 3285 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:37:51.726923 3288 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:37:51.726980 3277 server_base.cc:1034] running on GCE node
I20260807 18:37:51.727843 3277 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:37:51.729002 3277 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:37:51.730165 3277 hybrid_clock.cc:648] HybridClock initialized: now 1786127871730149 us; error 25 us; skew 500 ppm
Aug 07 18:37:51 dist-test-slave-6k22 krb5kdc[3179](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127871, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:37:51.733008 3277 init.cc:377] Logged in from keytab as kudu/127.11.14.125@KRBTEST.COM (short username kudu)
I20260807 18:37:51.733930 3277 webserver.cc:460] Webserver started at http://127.11.14.125:33761/ using document root <none> and password file <none>
I20260807 18:37:51.734442 3277 fs_manager.cc:359] Metadata directory not provided
I20260807 18:37:51.734500 3277 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:37:51.734663 3277 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:37:51.736665 3277 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786127579900523-11321-0/minicluster-data/master-1/data/instance:
uuid: "72de1af692c14c10afeee0316015bc78"
format_stamp: "Formatted at 2026-08-07 18:37:51 on dist-test-slave-6k22"
server_key: "09a045b72e1a81fa038a73913e5977f5"
server_key_iv: "d9789a37aacae7066b674e0396f87d55"
server_key_version: "encryptionkey@0"
I20260807 18:37:51.737236 3277 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786127579900523-11321-0/minicluster-data/master-1/wal/instance:
uuid: "72de1af692c14c10afeee0316015bc78"
format_stamp: "Formatted at 2026-08-07 18:37:51 on dist-test-slave-6k22"
server_key: "09a045b72e1a81fa038a73913e5977f5"
server_key_iv: "d9789a37aacae7066b674e0396f87d55"
server_key_version: "encryptionkey@0"
I20260807 18:37:51.740989 3277 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 18:37:51.743296 3295 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:37:51.744535 3277 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 18:37:51.744678 3277 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786127579900523-11321-0/minicluster-data/master-1/data,/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786127579900523-11321-0/minicluster-data/master-1/wal
uuid: "72de1af692c14c10afeee0316015bc78"
format_stamp: "Formatted at 2026-08-07 18:37:51 on dist-test-slave-6k22"
server_key: "09a045b72e1a81fa038a73913e5977f5"
server_key_iv: "d9789a37aacae7066b674e0396f87d55"
server_key_version: "encryptionkey@0"
I20260807 18:37:51.744786 3277 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786127579900523-11321-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786127579900523-11321-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786127579900523-11321-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:37:51.768848 3277 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:37:51.771839 3277 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:37:51.772044 3277 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:37:51.779059 3277 rpc_server.cc:307] RPC server started. Bound to: 127.11.14.125:37885
I20260807 18:37:51.779390 3355 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.14.125:37885 every 8 connection(s)
I20260807 18:37:51.780117 3277 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786127579900523-11321-0/minicluster-data/master-1/data/info.pb
I20260807 18:37:51.782958 3356 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:37:51.786875 3356 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.14.126" port: 45853 } has no permanent_uuid. Determining permanent_uuid...
I20260807 18:37:51.787871 11321 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkz56XN/build/debug/bin/kudu as pid 3277
I20260807 18:37:51.788024 11321 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786127579900523-11321-0/minicluster-data/master-1/wal/instance
I20260807 18:37:51.788287 11321 external_mini_cluster.cc:1428] Setting key 238a6f9d0430abd029a059bb14735ddf
I20260807 18:37:51.796465 3269 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.14.125" port: 37885 } attempt: 3
Aug 07 18:37:51 dist-test-slave-6k22 krb5kdc[3179](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127871, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.125@KRBTEST.COM for kudu/127.11.14.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.11.14.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.14.124@KRBTEST.COM" created.
I20260807 18:37:51.802712 3273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:51.788887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49688 (local address 127.11.14.126:45853)
0807 18:37:51.789060 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0807 18:37:51.789067 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:37:51.789923 (+ 856us) server_negotiation.cc:407] Connection header received
0807 18:37:51.790990 (+ 1067us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:37:51.790996 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:37:51.791068 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:37:51.791173 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:37:51.792631 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:51.793161 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:51.795017 (+ 1856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:51.795269 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:51.795353 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:51.798300 (+ 2947us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:37:51.798306 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:37:51.798310 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:37:51.798338 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:37:51.800121 (+ 1783us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:37:51.800717 (+ 596us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:37:51.800719 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:37:51.800721 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:37:51.800792 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:37:51.801160 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:37:51.801163 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:37:51.801165 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:37:51.801421 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:37:51.801531 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0807 18:37:51.802277 (+ 746us) server_negotiation.cc:299] Negotiation successful
0807 18:37:51.802466 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":5,"server-negotiator.queue_time_us":55,"spinlock_wait_cycles":768}
I20260807 18:37:51.803604 3358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:51.789214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:45853 (local address 127.0.0.1:49688)
0807 18:37:51.789725 (+ 511us) negotiation.cc:107] Waiting for socket to connect
0807 18:37:51.789787 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0807 18:37:51.790765 (+ 978us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:37:51.791323 (+ 558us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:37:51.791330 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:37:51.791768 (+ 438us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:37:51.792467 (+ 699us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:37:51.792490 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:37:51.793303 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:37:51.793310 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:37:51.794851 (+ 1541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:37:51.794865 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:37:51.795435 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:37:51.795443 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:37:51.795708 (+ 265us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:51.796343 (+ 635us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:37:51.796368 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:37:51.798131 (+ 1763us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:37:51.800244 (+ 2113us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:37:51.800249 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:37:51.800260 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:37:51.800618 (+ 358us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:37:51.800889 (+ 271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:37:51.800892 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:37:51.800894 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:37:51.801073 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:37:51.801544 (+ 471us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:37:51.801551 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:37:51.801938 (+ 387us) client_negotiation.cc:769] Sending connection context
0807 18:37:51.802156 (+ 218us) client_negotiation.cc:240] Negotiation successful
0807 18:37:51.802467 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":108,"threads_started":1}
I20260807 18:37:51.805325 3356 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.14.125" port: 37885 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 18:37:51 dist-test-slave-6k22 krb5kdc[3179](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127871, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.126@KRBTEST.COM for kudu/127.11.14.125@KRBTEST.COM
I20260807 18:37:51.810177 3274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:51.796942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.125:37885 (local address 127.0.0.1:42814)
0807 18:37:51.797066 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0807 18:37:51.797081 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 18:37:51.797162 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:37:51.797881 (+ 719us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:37:51.797885 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:37:51.798074 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:37:51.798196 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:37:51.798207 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:37:51.799492 (+ 1285us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:37:51.799497 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:37:51.800347 (+ 850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:37:51.800355 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:37:51.801141 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:37:51.801146 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:37:51.801331 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:51.801768 (+ 437us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:37:51.801778 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:37:51.806416 (+ 4638us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:37:51.808493 (+ 2077us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:37:51.808497 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:37:51.808499 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:37:51.808696 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:37:51.809005 (+ 309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:37:51.809008 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:37:51.809010 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:37:51.809095 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:37:51.809542 (+ 447us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:37:51.809545 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:37:51.809653 (+ 108us) client_negotiation.cc:769] Sending connection context
0807 18:37:51.809766 (+ 113us) client_negotiation.cc:240] Negotiation successful
0807 18:37:51.809918 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48,"mutex_wait_us":63}
I20260807 18:37:51.810449 3360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:51.797056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42814 (local address 127.11.14.125:37885)
0807 18:37:51.797399 (+ 343us) server_negotiation.cc:206] Beginning negotiation
0807 18:37:51.797413 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 18:37:51.797436 (+ 23us) server_negotiation.cc:407] Connection header received
0807 18:37:51.797480 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:37:51.797486 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:37:51.797687 (+ 201us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:37:51.797886 (+ 199us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:37:51.798326 (+ 440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:51.799359 (+ 1033us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:51.800453 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:51.801011 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:51.801126 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:51.806568 (+ 5442us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:37:51.806580 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:37:51.806594 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:37:51.806613 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:37:51.808375 (+ 1762us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:37:51.808819 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:37:51.808824 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:37:51.808826 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:37:51.808906 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:37:51.809186 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:37:51.809193 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:37:51.809195 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:37:51.809432 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:37:51.809573 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0807 18:37:51.809874 (+ 301us) server_negotiation.cc:299] Negotiation successful
0807 18:37:51.810031 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":237,"thread_start_us":160,"threads_started":1}
Aug 07 18:37:51 dist-test-slave-6k22 krb5kdc[3179](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127871, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.125@KRBTEST.COM for kudu/127.11.14.125@KRBTEST.COM
I20260807 18:37:51.812623 3269 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.14.124" port: 35709 } has no permanent_uuid. Determining permanent_uuid...
I20260807 18:37:51.813339 3274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:51.813100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.124:35709 (local address 127.0.0.1:51960)
0807 18:37:51.813213 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0807 18:37:51.813263 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.124:35709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260807 18:37:51.814399 3274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:51.813839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.124:35709 (local address 127.0.0.1:51964)
0807 18:37:51.814266 (+ 427us) negotiation.cc:107] Waiting for socket to connect
0807 18:37:51.814326 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.124:35709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":342}
W20260807 18:37:51.814548 3269 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.14.124:35709: Network error: Client connection negotiation failed: client connection to 127.11.14.124:35709: connect: Connection refused (error 111)
I20260807 18:37:51.816717 3358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:51.806370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.125:37885 (local address 127.0.0.1:42816)
0807 18:37:51.806535 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0807 18:37:51.806548 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 18:37:51.806609 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:37:51.807105 (+ 496us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:37:51.807110 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:37:51.807385 (+ 275us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:37:51.807596 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:37:51.807608 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:37:51.808743 (+ 1135us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:37:51.808750 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:37:51.809498 (+ 748us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:37:51.809508 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:37:51.810029 (+ 521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:37:51.810033 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:37:51.810124 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:51.810771 (+ 647us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:37:51.810782 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:37:51.812891 (+ 2109us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:37:51.814999 (+ 2108us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:37:51.815002 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:37:51.815003 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:37:51.815181 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:37:51.815497 (+ 316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:37:51.815499 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:37:51.815500 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:37:51.815852 (+ 352us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:37:51.816259 (+ 407us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:37:51.816265 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:37:51.816378 (+ 113us) client_negotiation.cc:769] Sending connection context
0807 18:37:51.816484 (+ 106us) client_negotiation.cc:240] Negotiation successful
0807 18:37:51.816602 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":98,"mutex_wait_us":21}
I20260807 18:37:51.816881 3362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:51.806429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42816 (local address 127.11.14.125:37885)
0807 18:37:51.806826 (+ 397us) server_negotiation.cc:206] Beginning negotiation
0807 18:37:51.806837 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 18:37:51.806852 (+ 15us) server_negotiation.cc:407] Connection header received
0807 18:37:51.806908 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:37:51.806912 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:37:51.806951 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:37:51.807347 (+ 396us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:37:51.807730 (+ 383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:51.808616 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:51.809603 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:51.809912 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:51.810335 (+ 423us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:51.813072 (+ 2737us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:37:51.813077 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:37:51.813080 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:37:51.813108 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:37:51.814878 (+ 1770us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:37:51.815270 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:37:51.815272 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:37:51.815273 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:37:51.815370 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:37:51.815928 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:37:51.815931 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:37:51.815932 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:37:51.816157 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:37:51.816237 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0807 18:37:51.816559 (+ 322us) server_negotiation.cc:299] Negotiation successful
0807 18:37:51.816670 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":332,"thread_start_us":120,"threads_started":1}
I20260807 18:37:51.817639 3356 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.14.124" port: 35709 } has no permanent_uuid. Determining permanent_uuid...
I20260807 18:37:51.818204 3358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:51.817988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.124:35709 (local address 127.0.0.1:51976)
0807 18:37:51.818090 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0807 18:37:51.818155 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.124:35709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260807 18:37:51.818455 3296 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.11.14.124:35709: connect: Connection refused (error 111)
I20260807 18:37:51.819247 3358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:51.819035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.124:35709 (local address 127.0.0.1:51988)
0807 18:37:51.819141 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0807 18:37:51.819187 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.124:35709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260807 18:37:51.819408 3356 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.14.124:35709: Network error: Client connection negotiation failed: client connection to 127.11.14.124:35709: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.14.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.124.keytab.
Entry for principal kudu/127.11.14.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.124.keytab.
I20260807 18:37:51.823390 11321 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkz56XN/build/debug/bin/kudu
/tmp/dist-test-taskkz56XN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786127579900523-11321-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786127579900523-11321-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786127579900523-11321-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786127579900523-11321-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.14.124:35709
--webserver_interface=127.11.14.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.124.keytab
--principal=kudu/127.11.14.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.14.84:40635
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.14.126:45853,127.11.14.125:37885,127.11.14.124:35709
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786127579900523-11321-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786127579900523-11321-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786127579900523-11321-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 18:37:51.845728 3269 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.14.124" port: 35709 } attempt: 1
I20260807 18:37:51.846794 3274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:51.846401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.124:35709 (local address 127.0.0.1:51996)
0807 18:37:51.846575 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0807 18:37:51.846703 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.124:35709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260807 18:37:51.847776 3274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:51.847382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.124:35709 (local address 127.0.0.1:52010)
0807 18:37:51.847550 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0807 18:37:51.847715 (+ 165us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.124:35709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260807 18:37:51.848001 3269 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.14.124:35709: Network error: Client connection negotiation failed: client connection to 127.11.14.124:35709: connect: Connection refused (error 111)
I20260807 18:37:51.868642 3356 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.14.124" port: 35709 } attempt: 1
I20260807 18:37:51.869572 3358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:51.869218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.124:35709 (local address 127.0.0.1:52012)
0807 18:37:51.869400 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0807 18:37:51.869472 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.124:35709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260807 18:37:51.870332 3358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:51.870079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.124:35709 (local address 127.0.0.1:52026)
0807 18:37:51.870190 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0807 18:37:51.870242 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.124:35709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260807 18:37:51.870474 3356 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.14.124:35709: Network error: Client connection negotiation failed: client connection to 127.11.14.124:35709: connect: Connection refused (error 111)
I20260807 18:37:51.923235 3269 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.14.124" port: 35709 } attempt: 2
I20260807 18:37:51.924403 3274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:51.924009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.124:35709 (local address 127.0.0.1:52038)
0807 18:37:51.924216 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0807 18:37:51.924300 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.124:35709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260807 18:37:51.925294 3274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:51.925014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.124:35709 (local address 127.0.0.1:52054)
0807 18:37:51.925154 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0807 18:37:51.925220 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.124:35709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260807 18:37:51.925490 3269 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.14.124:35709: Network error: Client connection negotiation failed: client connection to 127.11.14.124:35709: connect: Connection refused (error 111)
I20260807 18:37:51.938652 3356 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.14.124" port: 35709 } attempt: 2
I20260807 18:37:51.939594 3358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:51.939213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.124:35709 (local address 127.0.0.1:52070)
0807 18:37:51.939395 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0807 18:37:51.939472 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.124:35709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260807 18:37:51.940480 3358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:51.940180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.124:35709 (local address 127.0.0.1:52074)
0807 18:37:51.940342 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0807 18:37:51.940403 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.124:35709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
W20260807 18:37:51.940630 3356 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.14.124:35709: Network error: Client connection negotiation failed: client connection to 127.11.14.124:35709: connect: Connection refused (error 111)
W20260807 18:37:51.947240 3364 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 18:37:51.947537 3364 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:37:51.947640 3364 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:37:51.947708 3364 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:37:51.952636 3364 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:37:51.952694 3364 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:37:51.952716 3364 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:37:51.952734 3364 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:37:51.952751 3364 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:37:51.958568 3364 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.14.84:40635
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786127579900523-11321-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786127579900523-11321-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.11.14.126:45853,127.11.14.125:37885,127.11.14.124:35709
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.124.keytab
--principal=kudu/127.11.14.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.14.124:35709
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786127579900523-11321-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.11.14.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3364
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786127579900523-11321-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:37:51.959707 3364 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:37:51.960704 3364 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:37:51.969408 3370 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:37:51.969408 3373 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:37:51.969962 3371 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:37:51.970139 3364 server_base.cc:1034] running on GCE node
I20260807 18:37:51.970736 3364 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:37:51.971941 3364 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:37:51.973133 3364 hybrid_clock.cc:648] HybridClock initialized: now 1786127871973095 us; error 51 us; skew 500 ppm
Aug 07 18:37:51 dist-test-slave-6k22 krb5kdc[3179](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127871, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:37:51.976117 3364 init.cc:377] Logged in from keytab as kudu/127.11.14.124@KRBTEST.COM (short username kudu)
I20260807 18:37:51.977029 3364 webserver.cc:460] Webserver started at http://127.11.14.124:40683/ using document root <none> and password file <none>
I20260807 18:37:51.977628 3364 fs_manager.cc:359] Metadata directory not provided
I20260807 18:37:51.977731 3364 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:37:51.977980 3364 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:37:51.980170 3364 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786127579900523-11321-0/minicluster-data/master-2/data/instance:
uuid: "e310196022784c039b3fd3ef9f8f73ab"
format_stamp: "Formatted at 2026-08-07 18:37:51 on dist-test-slave-6k22"
server_key: "ec3953700734d228623eb76a2e78fbdd"
server_key_iv: "db39597b8ea0acdaf3491546fe044cb1"
server_key_version: "encryptionkey@0"
I20260807 18:37:51.980751 3364 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786127579900523-11321-0/minicluster-data/master-2/wal/instance:
uuid: "e310196022784c039b3fd3ef9f8f73ab"
format_stamp: "Formatted at 2026-08-07 18:37:51 on dist-test-slave-6k22"
server_key: "ec3953700734d228623eb76a2e78fbdd"
server_key_iv: "db39597b8ea0acdaf3491546fe044cb1"
server_key_version: "encryptionkey@0"
I20260807 18:37:51.984556 3364 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260807 18:37:51.986907 3380 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:37:51.988199 3364 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 18:37:51.988365 3364 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786127579900523-11321-0/minicluster-data/master-2/data,/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786127579900523-11321-0/minicluster-data/master-2/wal
uuid: "e310196022784c039b3fd3ef9f8f73ab"
format_stamp: "Formatted at 2026-08-07 18:37:51 on dist-test-slave-6k22"
server_key: "ec3953700734d228623eb76a2e78fbdd"
server_key_iv: "db39597b8ea0acdaf3491546fe044cb1"
server_key_version: "encryptionkey@0"
I20260807 18:37:51.988484 3364 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786127579900523-11321-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786127579900523-11321-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786127579900523-11321-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:37:52.005920 3364 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:37:52.009357 3364 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:37:52.009594 3364 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:37:52.016896 3364 rpc_server.cc:307] RPC server started. Bound to: 127.11.14.124:35709
I20260807 18:37:52.017186 3436 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.14.124:35709 every 8 connection(s)
I20260807 18:37:52.017989 3364 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786127579900523-11321-0/minicluster-data/master-2/data/info.pb
I20260807 18:37:52.020577 11321 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkz56XN/build/debug/bin/kudu as pid 3364
I20260807 18:37:52.020733 11321 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786127579900523-11321-0/minicluster-data/master-2/wal/instance
I20260807 18:37:52.021073 11321 external_mini_cluster.cc:1428] Setting key c613795a2d1ef80248149d400452d1f7
I20260807 18:37:52.021270 3437 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:37:52.024703 3269 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.14.124" port: 35709 } attempt: 3
I20260807 18:37:52.027060 3437 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.14.126" port: 45853 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 18:37:52 dist-test-slave-6k22 krb5kdc[3179](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127871, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.126@KRBTEST.COM for kudu/127.11.14.124@KRBTEST.COM
Aug 07 18:37:52 dist-test-slave-6k22 krb5kdc[3179](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127871, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.14.124@KRBTEST.COM
I20260807 18:37:52.032122 3356 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.14.124" port: 35709 } attempt: 3
Aug 07 18:37:52 dist-test-slave-6k22 krb5kdc[3179](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127871, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.124@KRBTEST.COM for kudu/127.11.14.126@KRBTEST.COM
I20260807 18:37:52.037642 3274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:52.025358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.124:35709 (local address 127.0.0.1:52098)
0807 18:37:52.025558 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0807 18:37:52.025574 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 18:37:52.025748 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:37:52.026261 (+ 513us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:37:52.026265 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:37:52.026452 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:37:52.026554 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:37:52.026560 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:37:52.027685 (+ 1125us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:37:52.027688 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:37:52.028248 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:37:52.028259 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:37:52.028846 (+ 587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:37:52.028849 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:37:52.028930 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:52.029628 (+ 698us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:37:52.029645 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:37:52.031963 (+ 2318us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:37:52.035187 (+ 3224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:37:52.035194 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:37:52.035197 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:37:52.035496 (+ 299us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:37:52.036004 (+ 508us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:37:52.036007 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:37:52.036009 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:37:52.036105 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:37:52.037088 (+ 983us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:37:52.037092 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:37:52.037197 (+ 105us) client_negotiation.cc:769] Sending connection context
0807 18:37:52.037294 (+ 97us) client_negotiation.cc:240] Negotiation successful
0807 18:37:52.037443 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120}
Aug 07 18:37:52 dist-test-slave-6k22 krb5kdc[3179](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127871, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.125@KRBTEST.COM for kudu/127.11.14.124@KRBTEST.COM
I20260807 18:37:52.039459 3442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:52.025431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52098 (local address 127.11.14.124:35709)
0807 18:37:52.026003 (+ 572us) server_negotiation.cc:206] Beginning negotiation
0807 18:37:52.026013 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 18:37:52.026027 (+ 14us) server_negotiation.cc:407] Connection header received
0807 18:37:52.026063 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:37:52.026066 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:37:52.026107 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:37:52.026203 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:37:52.026694 (+ 491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:52.027534 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:52.028382 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:52.028698 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:52.029871 (+ 1173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:52.032159 (+ 2288us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:37:52.032166 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:37:52.032177 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:37:52.032208 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:37:52.035026 (+ 2818us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:37:52.035686 (+ 660us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:37:52.035689 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:37:52.035696 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:37:52.035877 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:37:52.036237 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:37:52.036241 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:37:52.036242 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:37:52.036942 (+ 700us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:37:52.037949 (+ 1007us) server_negotiation.cc:1035] Waiting for connection context
0807 18:37:52.038043 (+ 94us) server_negotiation.cc:299] Negotiation successful
0807 18:37:52.038380 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":201,"server-negotiator.queue_time_us":502,"spinlock_wait_cycles":65408,"thread_start_us":149,"threads_started":1}
I20260807 18:37:52.039459 3440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:52.022549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52088 (local address 127.11.14.124:35709)
0807 18:37:52.023020 (+ 471us) server_negotiation.cc:206] Beginning negotiation
0807 18:37:52.023036 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 18:37:52.023073 (+ 37us) server_negotiation.cc:407] Connection header received
0807 18:37:52.023900 (+ 827us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:37:52.023920 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:37:52.024244 (+ 324us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:37:52.024596 (+ 352us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:37:52.025573 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:52.026859 (+ 1286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:52.027841 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:52.028590 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:52.028798 (+ 208us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:52.032453 (+ 3655us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:37:52.032461 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:37:52.032467 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:37:52.032501 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:37:52.035074 (+ 2573us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:37:52.035682 (+ 608us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:37:52.035689 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:37:52.035696 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:37:52.035871 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:37:52.036241 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:37:52.036244 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:37:52.036246 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:37:52.036942 (+ 696us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:37:52.037727 (+ 785us) server_negotiation.cc:1035] Waiting for connection context
0807 18:37:52.037922 (+ 195us) server_negotiation.cc:299] Negotiation successful
0807 18:37:52.038314 (+ 392us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":115,"server-negotiator.queue_time_us":329,"spinlock_wait_cycles":12160,"thread_start_us":146,"threads_started":1}
I20260807 18:37:52.042030 3273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:52.029171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49702 (local address 127.11.14.126:45853)
0807 18:37:52.029316 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0807 18:37:52.029321 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:37:52.029638 (+ 317us) server_negotiation.cc:407] Connection header received
0807 18:37:52.029842 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:37:52.029845 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:37:52.029890 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:37:52.029946 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:37:52.031102 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:52.031708 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:52.033772 (+ 2064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:52.034100 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:52.034175 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:52.037727 (+ 3552us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:37:52.037734 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:37:52.037740 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:37:52.037767 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:37:52.039539 (+ 1772us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:37:52.040374 (+ 835us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:37:52.040377 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:37:52.040379 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:37:52.040469 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:37:52.040824 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:37:52.040828 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:37:52.040830 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:37:52.041144 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:37:52.041232 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0807 18:37:52.041752 (+ 520us) server_negotiation.cc:299] Negotiation successful
0807 18:37:52.041887 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260807 18:37:52.042578 3445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:52.029153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:45853 (local address 127.0.0.1:49702)
0807 18:37:52.029504 (+ 351us) negotiation.cc:107] Waiting for socket to connect
0807 18:37:52.029552 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0807 18:37:52.029724 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:37:52.030124 (+ 400us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:37:52.030130 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:37:52.030781 (+ 651us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:37:52.030932 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:37:52.030943 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:37:52.031852 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:37:52.031855 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:37:52.033227 (+ 1372us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:37:52.033241 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:37:52.034341 (+ 1100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:37:52.034345 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:37:52.034454 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:52.035549 (+ 1095us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:37:52.035585 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:37:52.037559 (+ 1974us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:37:52.039908 (+ 2349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:37:52.039914 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:37:52.039920 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:37:52.040244 (+ 324us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:37:52.040598 (+ 354us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:37:52.040601 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:37:52.040603 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:37:52.040705 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:37:52.041323 (+ 618us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:37:52.041329 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:37:52.041455 (+ 126us) client_negotiation.cc:769] Sending connection context
0807 18:37:52.042176 (+ 721us) client_negotiation.cc:240] Negotiation successful
0807 18:37:52.042376 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":193,"spinlock_wait_cycles":10624,"thread_start_us":87,"threads_started":1}
I20260807 18:37:52.043821 3358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:52.032585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.124:35709 (local address 127.0.0.1:52112)
0807 18:37:52.032756 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0807 18:37:52.032773 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 18:37:52.032861 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:37:52.033498 (+ 637us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:37:52.033503 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:37:52.033712 (+ 209us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:37:52.033827 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:37:52.033834 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:37:52.034831 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:37:52.034836 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:37:52.035786 (+ 950us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:37:52.035800 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:37:52.036490 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:37:52.036497 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:37:52.036607 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:52.037074 (+ 467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:37:52.037093 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:37:52.039144 (+ 2051us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:37:52.041629 (+ 2485us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:37:52.041632 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:37:52.041634 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:37:52.041880 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:37:52.042442 (+ 562us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:37:52.042446 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:37:52.042448 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:37:52.042546 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:37:52.043241 (+ 695us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:37:52.043244 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:37:52.043353 (+ 109us) client_negotiation.cc:769] Sending connection context
0807 18:37:52.043488 (+ 135us) client_negotiation.cc:240] Negotiation successful
0807 18:37:52.043664 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":66}
I20260807 18:37:52.043908 3446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:52.032884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52112 (local address 127.11.14.124:35709)
0807 18:37:52.033226 (+ 342us) server_negotiation.cc:206] Beginning negotiation
0807 18:37:52.033236 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 18:37:52.033251 (+ 15us) server_negotiation.cc:407] Connection header received
0807 18:37:52.033291 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:37:52.033296 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:37:52.033350 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:37:52.033450 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:37:52.033953 (+ 503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:52.034686 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:52.035943 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:52.036349 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:52.036824 (+ 475us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:52.039303 (+ 2479us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:37:52.039308 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:37:52.039314 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:37:52.039354 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:37:52.041494 (+ 2140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:37:52.042007 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:37:52.042010 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:37:52.042012 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:37:52.042308 (+ 296us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:37:52.042885 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:37:52.042890 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:37:52.042892 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:37:52.043109 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:37:52.043211 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0807 18:37:52.043594 (+ 383us) server_negotiation.cc:299] Negotiation successful
0807 18:37:52.043728 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":259,"thread_start_us":86,"threads_started":1}
I20260807 18:37:52.044574 3269 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ac07d9da9d3f460caecd1af62b86a26f: Bootstrap starting.
I20260807 18:37:52.047711 3437 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.14.125" port: 37885 } has no permanent_uuid. Determining permanent_uuid...
I20260807 18:37:52.047748 3269 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ac07d9da9d3f460caecd1af62b86a26f: Neither blocks nor log segments found. Creating new log.
Aug 07 18:37:52 dist-test-slave-6k22 krb5kdc[3179](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127871, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.14.126@KRBTEST.COM
I20260807 18:37:52.049610 3269 log.cc:824] T 00000000000000000000000000000000 P ac07d9da9d3f460caecd1af62b86a26f: Log is configured to *not* fsync() on all Append() calls
I20260807 18:37:52.050521 3356 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 72de1af692c14c10afeee0316015bc78: Bootstrap starting.
I20260807 18:37:52.052134 3269 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ac07d9da9d3f460caecd1af62b86a26f: No bootstrap required, opened a new log
Aug 07 18:37:52 dist-test-slave-6k22 krb5kdc[3179](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127871, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.124@KRBTEST.COM for kudu/127.11.14.125@KRBTEST.COM
I20260807 18:37:52.053673 3273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:52.042523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49716 (local address 127.11.14.126:45853)
0807 18:37:52.043091 (+ 568us) server_negotiation.cc:206] Beginning negotiation
0807 18:37:52.043095 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:37:52.043108 (+ 13us) server_negotiation.cc:407] Connection header received
0807 18:37:52.043144 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:37:52.043149 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:37:52.043197 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:37:52.043270 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:37:52.044352 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:52.044859 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:52.045975 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:52.046166 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:52.046240 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:52.049316 (+ 3076us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:37:52.049320 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:37:52.049323 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:37:52.049347 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:37:52.051064 (+ 1717us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:37:52.051695 (+ 631us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:37:52.051699 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:37:52.051701 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:37:52.051799 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:37:52.052146 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:37:52.052153 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:37:52.052157 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:37:52.052448 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:37:52.052633 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0807 18:37:52.053327 (+ 694us) server_negotiation.cc:299] Negotiation successful
0807 18:37:52.053511 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":497}
I20260807 18:37:52.053757 3356 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 72de1af692c14c10afeee0316015bc78: Neither blocks nor log segments found. Creating new log.
I20260807 18:37:52.055465 3356 log.cc:824] T 00000000000000000000000000000000 P 72de1af692c14c10afeee0316015bc78: Log is configured to *not* fsync() on all Append() calls
I20260807 18:37:52.057605 3362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:52.048408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42822 (local address 127.11.14.125:37885)
0807 18:37:52.048535 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0807 18:37:52.048540 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:37:52.048555 (+ 15us) server_negotiation.cc:407] Connection header received
0807 18:37:52.048593 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:37:52.048596 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:37:52.048656 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:37:52.048753 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:37:52.049352 (+ 599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:52.050101 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:52.051166 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:52.051501 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:52.051614 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:52.053936 (+ 2322us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:37:52.053939 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:37:52.053941 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:37:52.053969 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:37:52.055728 (+ 1759us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:37:52.056233 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:37:52.056240 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:37:52.056244 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:37:52.056338 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:37:52.056678 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:37:52.056681 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:37:52.056683 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:37:52.056907 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:37:52.056979 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0807 18:37:52.057351 (+ 372us) server_negotiation.cc:299] Negotiation successful
0807 18:37:52.057463 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260807 18:37:52.057528 3269 raft_consensus.cc:348] T 00000000000000000000000000000000 P ac07d9da9d3f460caecd1af62b86a26f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac07d9da9d3f460caecd1af62b86a26f" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 45853 } } peers { permanent_uuid: "72de1af692c14c10afeee0316015bc78" member_type: VOTER last_known_addr { host: "127.11.14.125" port: 37885 } } peers { permanent_uuid: "e310196022784c039b3fd3ef9f8f73ab" member_type: VOTER last_known_addr { host: "127.11.14.124" port: 35709 } }
I20260807 18:37:52.057796 3356 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 72de1af692c14c10afeee0316015bc78: No bootstrap required, opened a new log
I20260807 18:37:52.057868 3269 raft_consensus.cc:729] T 00000000000000000000000000000000 P ac07d9da9d3f460caecd1af62b86a26f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac07d9da9d3f460caecd1af62b86a26f, State: Initialized, Role: FOLLOWER
I20260807 18:37:52.058004 3445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:52.048182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.125:37885 (local address 127.0.0.1:42822)
0807 18:37:52.048412 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0807 18:37:52.048426 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 18:37:52.048499 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:37:52.048782 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:37:52.048785 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:37:52.048961 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:37:52.049049 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:37:52.049055 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:37:52.050247 (+ 1192us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:37:52.050252 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:37:52.051025 (+ 773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:37:52.051036 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:37:52.051784 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:37:52.051788 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:37:52.051877 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:52.052316 (+ 439us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:37:52.052340 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:37:52.053772 (+ 1432us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:37:52.055850 (+ 2078us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:37:52.055853 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:37:52.055854 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:37:52.056108 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:37:52.056469 (+ 361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:37:52.056471 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:37:52.056473 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:37:52.056570 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:37:52.057053 (+ 483us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:37:52.057056 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:37:52.057150 (+ 94us) client_negotiation.cc:769] Sending connection context
0807 18:37:52.057731 (+ 581us) client_negotiation.cc:240] Negotiation successful
0807 18:37:52.057867 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167}
I20260807 18:37:52.058688 3437 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.14.124" port: 35709 } has no permanent_uuid. Determining permanent_uuid...
I20260807 18:37:52.058485 3269 consensus_queue.cc:260] T 00000000000000000000000000000000 P ac07d9da9d3f460caecd1af62b86a26f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac07d9da9d3f460caecd1af62b86a26f" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 45853 } } peers { permanent_uuid: "72de1af692c14c10afeee0316015bc78" member_type: VOTER last_known_addr { host: "127.11.14.125" port: 37885 } } peers { permanent_uuid: "e310196022784c039b3fd3ef9f8f73ab" member_type: VOTER last_known_addr { host: "127.11.14.124" port: 35709 } }
I20260807 18:37:52.060992 3269 sys_catalog.cc:564] T 00000000000000000000000000000000 P ac07d9da9d3f460caecd1af62b86a26f [sys.catalog]: configured and running, proceeding with master startup.
Aug 07 18:37:52 dist-test-slave-6k22 krb5kdc[3179](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127871, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.14.125@KRBTEST.COM
Aug 07 18:37:52 dist-test-slave-6k22 krb5kdc[3179](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127871, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.124@KRBTEST.COM for kudu/127.11.14.124@KRBTEST.COM
I20260807 18:37:52.064733 3449 sys_catalog.cc:455] T 00000000000000000000000000000000 P ac07d9da9d3f460caecd1af62b86a26f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac07d9da9d3f460caecd1af62b86a26f" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 45853 } } peers { permanent_uuid: "72de1af692c14c10afeee0316015bc78" member_type: VOTER last_known_addr { host: "127.11.14.125" port: 37885 } } peers { permanent_uuid: "e310196022784c039b3fd3ef9f8f73ab" member_type: VOTER last_known_addr { host: "127.11.14.124" port: 35709 } } }
I20260807 18:37:52.064903 3449 sys_catalog.cc:458] T 00000000000000000000000000000000 P ac07d9da9d3f460caecd1af62b86a26f [sys.catalog]: This master's current role is: FOLLOWER
I20260807 18:37:52.068212 3446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:52.059319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52118 (local address 127.11.14.124:35709)
0807 18:37:52.059411 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0807 18:37:52.059415 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:37:52.059428 (+ 13us) server_negotiation.cc:407] Connection header received
0807 18:37:52.059462 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:37:52.059464 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:37:52.059505 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:37:52.059648 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:37:52.060093 (+ 445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:52.060835 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:52.061861 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:52.062207 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:52.062281 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:52.064764 (+ 2483us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:37:52.064767 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:37:52.064770 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:37:52.064793 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:37:52.066485 (+ 1692us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:37:52.066927 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:37:52.066932 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:37:52.066934 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:37:52.067013 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:37:52.067302 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:37:52.067305 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:37:52.067306 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:37:52.067512 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:37:52.067616 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0807 18:37:52.067941 (+ 325us) server_negotiation.cc:299] Negotiation successful
0807 18:37:52.068058 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260807 18:37:52.068426 3360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:52.055598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42828 (local address 127.11.14.125:37885)
0807 18:37:52.055722 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0807 18:37:52.055726 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:37:52.055741 (+ 15us) server_negotiation.cc:407] Connection header received
0807 18:37:52.055899 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:37:52.055903 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:37:52.055948 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:37:52.056014 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:37:52.056928 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:52.057757 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:52.059676 (+ 1919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:52.060023 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:52.060103 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:52.063475 (+ 3372us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:37:52.063479 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:37:52.063482 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:37:52.063511 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:37:52.066190 (+ 2679us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:37:52.066735 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:37:52.066739 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:37:52.066741 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:37:52.066837 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:37:52.067247 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:37:52.067251 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:37:52.067253 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:37:52.067473 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:37:52.067708 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0807 18:37:52.068106 (+ 398us) server_negotiation.cc:299] Negotiation successful
0807 18:37:52.068236 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260807 18:37:52.068675 3445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:52.059167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.124:35709 (local address 127.0.0.1:52118)
0807 18:37:52.059266 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0807 18:37:52.059283 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 18:37:52.059368 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:37:52.059654 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:37:52.059658 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:37:52.059841 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:37:52.059954 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:37:52.059963 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:37:52.060955 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:37:52.060958 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:37:52.061733 (+ 775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:37:52.061744 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:37:52.062423 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:37:52.062428 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:37:52.062506 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:52.062967 (+ 461us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:37:52.062981 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:37:52.064638 (+ 1657us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:37:52.066593 (+ 1955us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:37:52.066596 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:37:52.066597 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:37:52.066824 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:37:52.067119 (+ 295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:37:52.067125 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:37:52.067128 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:37:52.067211 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:37:52.067680 (+ 469us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:37:52.067682 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:37:52.067769 (+ 87us) client_negotiation.cc:769] Sending connection context
0807 18:37:52.068397 (+ 628us) client_negotiation.cc:240] Negotiation successful
0807 18:37:52.068523 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
W20260807 18:37:52.069965 3461 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ac07d9da9d3f460caecd1af62b86a26f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 18:37:52.070071 3461 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 18:37:52.067842 3194 master_runner.cc:186] Error getting master registration for 127.11.14.125:37885: OK, instance_id { permanent_uuid: "72de1af692c14c10afeee0316015bc78" instance_seqno: 1786127871777092 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 18:37:52.071282 3194 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 18:37:52.063750 3356 raft_consensus.cc:348] T 00000000000000000000000000000000 P 72de1af692c14c10afeee0316015bc78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac07d9da9d3f460caecd1af62b86a26f" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 45853 } } peers { permanent_uuid: "72de1af692c14c10afeee0316015bc78" member_type: VOTER last_known_addr { host: "127.11.14.125" port: 37885 } } peers { permanent_uuid: "e310196022784c039b3fd3ef9f8f73ab" member_type: VOTER last_known_addr { host: "127.11.14.124" port: 35709 } }
I20260807 18:37:52.071756 3356 raft_consensus.cc:729] T 00000000000000000000000000000000 P 72de1af692c14c10afeee0316015bc78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72de1af692c14c10afeee0316015bc78, State: Initialized, Role: FOLLOWER
I20260807 18:37:52.071928 3437 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e310196022784c039b3fd3ef9f8f73ab: Bootstrap starting.
I20260807 18:37:52.072335 3356 consensus_queue.cc:260] T 00000000000000000000000000000000 P 72de1af692c14c10afeee0316015bc78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac07d9da9d3f460caecd1af62b86a26f" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 45853 } } peers { permanent_uuid: "72de1af692c14c10afeee0316015bc78" member_type: VOTER last_known_addr { host: "127.11.14.125" port: 37885 } } peers { permanent_uuid: "e310196022784c039b3fd3ef9f8f73ab" member_type: VOTER last_known_addr { host: "127.11.14.124" port: 35709 } }
I20260807 18:37:52.074556 3356 sys_catalog.cc:564] T 00000000000000000000000000000000 P 72de1af692c14c10afeee0316015bc78 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:37:52.074716 3437 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e310196022784c039b3fd3ef9f8f73ab: Neither blocks nor log segments found. Creating new log.
I20260807 18:37:52.075022 11321 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 18:37:52.075672 3437 log.cc:824] T 00000000000000000000000000000000 P e310196022784c039b3fd3ef9f8f73ab: Log is configured to *not* fsync() on all Append() calls
I20260807 18:37:52.076601 3239 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "ac07d9da9d3f460caecd1af62b86a26f"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49716
I20260807 18:37:52.076963 3239 raft_consensus.cc:482] T 00000000000000000000000000000000 P ac07d9da9d3f460caecd1af62b86a26f [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 18:37:52.077119 3239 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ac07d9da9d3f460caecd1af62b86a26f [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:37:52.078181 3463 sys_catalog.cc:455] T 00000000000000000000000000000000 P 72de1af692c14c10afeee0316015bc78 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac07d9da9d3f460caecd1af62b86a26f" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 45853 } } peers { permanent_uuid: "72de1af692c14c10afeee0316015bc78" member_type: VOTER last_known_addr { host: "127.11.14.125" port: 37885 } } peers { permanent_uuid: "e310196022784c039b3fd3ef9f8f73ab" member_type: VOTER last_known_addr { host: "127.11.14.124" port: 35709 } } }
I20260807 18:37:52.078320 3463 sys_catalog.cc:458] T 00000000000000000000000000000000 P 72de1af692c14c10afeee0316015bc78 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 18:37:52.078596 3239 raft_consensus.cc:504] T 00000000000000000000000000000000 P ac07d9da9d3f460caecd1af62b86a26f [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac07d9da9d3f460caecd1af62b86a26f" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 45853 } } peers { permanent_uuid: "72de1af692c14c10afeee0316015bc78" member_type: VOTER last_known_addr { host: "127.11.14.125" port: 37885 } } peers { permanent_uuid: "e310196022784c039b3fd3ef9f8f73ab" member_type: VOTER last_known_addr { host: "127.11.14.124" port: 35709 } }
I20260807 18:37:52.080147 3437 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e310196022784c039b3fd3ef9f8f73ab: No bootstrap required, opened a new log
I20260807 18:37:52.080087 3326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ac07d9da9d3f460caecd1af62b86a26f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "72de1af692c14c10afeee0316015bc78"
I20260807 18:37:52.080300 3326 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 72de1af692c14c10afeee0316015bc78 [term 0 FOLLOWER]: Advancing to term 1
W20260807 18:37:52.080739 3477 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 72de1af692c14c10afeee0316015bc78: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 18:37:52.080881 3477 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 18:37:52.081629 3326 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 72de1af692c14c10afeee0316015bc78 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ac07d9da9d3f460caecd1af62b86a26f in term 1.
I20260807 18:37:52.083447 3212 leader_election.cc:302] T 00000000000000000000000000000000 P ac07d9da9d3f460caecd1af62b86a26f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 72de1af692c14c10afeee0316015bc78, ac07d9da9d3f460caecd1af62b86a26f; no voters:
I20260807 18:37:52.083791 3449 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ac07d9da9d3f460caecd1af62b86a26f [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:37:52.084115 3449 raft_consensus.cc:686] T 00000000000000000000000000000000 P ac07d9da9d3f460caecd1af62b86a26f [term 1 LEADER]: Becoming Leader. State: Replica: ac07d9da9d3f460caecd1af62b86a26f, State: Running, Role: LEADER
I20260807 18:37:52.084550 3449 consensus_queue.cc:237] T 00000000000000000000000000000000 P ac07d9da9d3f460caecd1af62b86a26f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac07d9da9d3f460caecd1af62b86a26f" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 45853 } } peers { permanent_uuid: "72de1af692c14c10afeee0316015bc78" member_type: VOTER last_known_addr { host: "127.11.14.125" port: 37885 } } peers { permanent_uuid: "e310196022784c039b3fd3ef9f8f73ab" member_type: VOTER last_known_addr { host: "127.11.14.124" port: 35709 } }
I20260807 18:37:52.085050 3239 leader_election.cc:288] T 00000000000000000000000000000000 P ac07d9da9d3f460caecd1af62b86a26f [CANDIDATE]: Term 1 election: Requested vote from peers 72de1af692c14c10afeee0316015bc78 (127.11.14.125:37885), e310196022784c039b3fd3ef9f8f73ab (127.11.14.124:35709)
I20260807 18:37:52.085109 3437 raft_consensus.cc:348] T 00000000000000000000000000000000 P e310196022784c039b3fd3ef9f8f73ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac07d9da9d3f460caecd1af62b86a26f" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 45853 } } peers { permanent_uuid: "72de1af692c14c10afeee0316015bc78" member_type: VOTER last_known_addr { host: "127.11.14.125" port: 37885 } } peers { permanent_uuid: "e310196022784c039b3fd3ef9f8f73ab" member_type: VOTER last_known_addr { host: "127.11.14.124" port: 35709 } }
I20260807 18:37:52.085366 3437 raft_consensus.cc:729] T 00000000000000000000000000000000 P e310196022784c039b3fd3ef9f8f73ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e310196022784c039b3fd3ef9f8f73ab, State: Initialized, Role: FOLLOWER
I20260807 18:37:52.085709 3437 consensus_queue.cc:260] T 00000000000000000000000000000000 P e310196022784c039b3fd3ef9f8f73ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac07d9da9d3f460caecd1af62b86a26f" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 45853 } } peers { permanent_uuid: "72de1af692c14c10afeee0316015bc78" member_type: VOTER last_known_addr { host: "127.11.14.125" port: 37885 } } peers { permanent_uuid: "e310196022784c039b3fd3ef9f8f73ab" member_type: VOTER last_known_addr { host: "127.11.14.124" port: 35709 } }
I20260807 18:37:52.087137 3437 sys_catalog.cc:564] T 00000000000000000000000000000000 P e310196022784c039b3fd3ef9f8f73ab [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:37:52.087286 3406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ac07d9da9d3f460caecd1af62b86a26f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "e310196022784c039b3fd3ef9f8f73ab"
I20260807 18:37:52.087504 3406 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e310196022784c039b3fd3ef9f8f73ab [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:37:52.088972 3406 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e310196022784c039b3fd3ef9f8f73ab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ac07d9da9d3f460caecd1af62b86a26f in term 1.
I20260807 18:37:52.089756 3449 sys_catalog.cc:455] T 00000000000000000000000000000000 P ac07d9da9d3f460caecd1af62b86a26f [sys.catalog]: SysCatalogTable state changed. Reason: New leader ac07d9da9d3f460caecd1af62b86a26f. Latest consensus state: current_term: 1 leader_uuid: "ac07d9da9d3f460caecd1af62b86a26f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac07d9da9d3f460caecd1af62b86a26f" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 45853 } } peers { permanent_uuid: "72de1af692c14c10afeee0316015bc78" member_type: VOTER last_known_addr { host: "127.11.14.125" port: 37885 } } peers { permanent_uuid: "e310196022784c039b3fd3ef9f8f73ab" member_type: VOTER last_known_addr { host: "127.11.14.124" port: 35709 } } }
I20260807 18:37:52.089936 3449 sys_catalog.cc:458] T 00000000000000000000000000000000 P ac07d9da9d3f460caecd1af62b86a26f [sys.catalog]: This master's current role is: LEADER
I20260807 18:37:52.093209 3480 sys_catalog.cc:455] T 00000000000000000000000000000000 P e310196022784c039b3fd3ef9f8f73ab [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac07d9da9d3f460caecd1af62b86a26f" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 45853 } } peers { permanent_uuid: "72de1af692c14c10afeee0316015bc78" member_type: VOTER last_known_addr { host: "127.11.14.125" port: 37885 } } peers { permanent_uuid: "e310196022784c039b3fd3ef9f8f73ab" member_type: VOTER last_known_addr { host: "127.11.14.124" port: 35709 } } }
I20260807 18:37:52.093432 3480 sys_catalog.cc:458] T 00000000000000000000000000000000 P e310196022784c039b3fd3ef9f8f73ab [sys.catalog]: This master's current role is: FOLLOWER
I20260807 18:37:52.090742 3277 master_runner.cc:186] Error getting master registration for 127.11.14.124:35709: OK, instance_id { permanent_uuid: "e310196022784c039b3fd3ef9f8f73ab" instance_seqno: 1786127872014751 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 18:37:52.094908 3277 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 18:37:52.095646 3486 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:37:52.096765 3486 catalog_manager.cc:1484] Initializing Kudu cluster ID...
W20260807 18:37:52.105822 3500 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e310196022784c039b3fd3ef9f8f73ab: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260807 18:37:52.110889 3326 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 72de1af692c14c10afeee0316015bc78 [term 1 FOLLOWER]: Refusing update from remote peer ac07d9da9d3f460caecd1af62b86a26f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260807 18:37:52.111159 3500 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 18:37:52.109903 3406 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e310196022784c039b3fd3ef9f8f73ab [term 1 FOLLOWER]: Refusing update from remote peer ac07d9da9d3f460caecd1af62b86a26f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:37:52.111754 3479 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ac07d9da9d3f460caecd1af62b86a26f [LEADER]: Connected to new peer: Peer: permanent_uuid: "72de1af692c14c10afeee0316015bc78" member_type: VOTER last_known_addr { host: "127.11.14.125" port: 37885 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:37:52.112035 3449 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ac07d9da9d3f460caecd1af62b86a26f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e310196022784c039b3fd3ef9f8f73ab" member_type: VOTER last_known_addr { host: "127.11.14.124" port: 35709 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:37:52.116099 3463 sys_catalog.cc:455] T 00000000000000000000000000000000 P 72de1af692c14c10afeee0316015bc78 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ac07d9da9d3f460caecd1af62b86a26f. Latest consensus state: current_term: 1 leader_uuid: "ac07d9da9d3f460caecd1af62b86a26f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac07d9da9d3f460caecd1af62b86a26f" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 45853 } } peers { permanent_uuid: "72de1af692c14c10afeee0316015bc78" member_type: VOTER last_known_addr { host: "127.11.14.125" port: 37885 } } peers { permanent_uuid: "e310196022784c039b3fd3ef9f8f73ab" member_type: VOTER last_known_addr { host: "127.11.14.124" port: 35709 } } }
I20260807 18:37:52.116220 3463 sys_catalog.cc:458] T 00000000000000000000000000000000 P 72de1af692c14c10afeee0316015bc78 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 18:37:52.116248 3480 sys_catalog.cc:455] T 00000000000000000000000000000000 P e310196022784c039b3fd3ef9f8f73ab [sys.catalog]: SysCatalogTable state changed. Reason: New leader ac07d9da9d3f460caecd1af62b86a26f. Latest consensus state: current_term: 1 leader_uuid: "ac07d9da9d3f460caecd1af62b86a26f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac07d9da9d3f460caecd1af62b86a26f" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 45853 } } peers { permanent_uuid: "72de1af692c14c10afeee0316015bc78" member_type: VOTER last_known_addr { host: "127.11.14.125" port: 37885 } } peers { permanent_uuid: "e310196022784c039b3fd3ef9f8f73ab" member_type: VOTER last_known_addr { host: "127.11.14.124" port: 35709 } } }
I20260807 18:37:52.116397 3480 sys_catalog.cc:458] T 00000000000000000000000000000000 P e310196022784c039b3fd3ef9f8f73ab [sys.catalog]: This master's current role is: FOLLOWER
I20260807 18:37:52.118294 3446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:52.095067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52120 (local address 127.11.14.124:35709)
0807 18:37:52.095181 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0807 18:37:52.095186 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:37:52.100012 (+ 4826us) server_negotiation.cc:407] Connection header received
0807 18:37:52.100179 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:37:52.100183 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:37:52.100231 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:37:52.100285 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:37:52.101668 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:52.103609 (+ 1941us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:52.104947 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:52.105317 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:52.105395 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:52.109635 (+ 4240us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:37:52.109639 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:37:52.109642 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:37:52.109669 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:37:52.111692 (+ 2023us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:37:52.116920 (+ 5228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:37:52.116924 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:37:52.116926 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:37:52.117015 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:37:52.117309 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:37:52.117312 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:37:52.117314 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:37:52.117543 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:37:52.117669 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0807 18:37:52.118020 (+ 351us) server_negotiation.cc:299] Negotiation successful
0807 18:37:52.118139 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260807 18:37:52.120148 3360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:52.092091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42830 (local address 127.11.14.125:37885)
0807 18:37:52.092237 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0807 18:37:52.092242 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:37:52.099313 (+ 7071us) server_negotiation.cc:407] Connection header received
0807 18:37:52.099492 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:37:52.099496 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:37:52.099576 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:37:52.099667 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:37:52.108434 (+ 8767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:52.109190 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:52.110376 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:52.110751 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:52.110836 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:52.115918 (+ 5082us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:37:52.115922 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:37:52.115925 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:37:52.115957 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:37:52.117954 (+ 1997us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:37:52.118537 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:37:52.118541 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:37:52.118543 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:37:52.118634 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:37:52.118952 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:37:52.118956 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:37:52.118958 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:37:52.119198 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:37:52.119340 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0807 18:37:52.119862 (+ 522us) server_negotiation.cc:299] Negotiation successful
0807 18:37:52.119988 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260807 18:37:52.127924 3273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:52.095259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49732 (local address 127.11.14.126:45853)
0807 18:37:52.095371 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0807 18:37:52.095376 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:37:52.100012 (+ 4636us) server_negotiation.cc:407] Connection header received
0807 18:37:52.100058 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:37:52.100061 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:37:52.100107 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:37:52.100184 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:37:52.107688 (+ 7504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:52.108460 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:52.110017 (+ 1557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:52.110402 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:52.110504 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:52.123744 (+ 13240us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:37:52.123749 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:37:52.123751 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:37:52.123782 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:37:52.125709 (+ 1927us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:37:52.126214 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:37:52.126218 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:37:52.126220 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:37:52.126310 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:37:52.126606 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:37:52.126609 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:37:52.126611 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:37:52.126820 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:37:52.126937 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0807 18:37:52.127648 (+ 711us) server_negotiation.cc:299] Negotiation successful
0807 18:37:52.127770 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260807 18:37:52.132405 3449 sys_catalog.cc:455] T 00000000000000000000000000000000 P ac07d9da9d3f460caecd1af62b86a26f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ac07d9da9d3f460caecd1af62b86a26f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac07d9da9d3f460caecd1af62b86a26f" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 45853 } } peers { permanent_uuid: "72de1af692c14c10afeee0316015bc78" member_type: VOTER last_known_addr { host: "127.11.14.125" port: 37885 } } peers { permanent_uuid: "e310196022784c039b3fd3ef9f8f73ab" member_type: VOTER last_known_addr { host: "127.11.14.124" port: 35709 } } }
I20260807 18:37:52.132535 3449 sys_catalog.cc:458] T 00000000000000000000000000000000 P ac07d9da9d3f460caecd1af62b86a26f [sys.catalog]: This master's current role is: LEADER
I20260807 18:37:52.133098 3449 sys_catalog.cc:455] T 00000000000000000000000000000000 P ac07d9da9d3f460caecd1af62b86a26f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ac07d9da9d3f460caecd1af62b86a26f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac07d9da9d3f460caecd1af62b86a26f" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 45853 } } peers { permanent_uuid: "72de1af692c14c10afeee0316015bc78" member_type: VOTER last_known_addr { host: "127.11.14.125" port: 37885 } } peers { permanent_uuid: "e310196022784c039b3fd3ef9f8f73ab" member_type: VOTER last_known_addr { host: "127.11.14.124" port: 35709 } } }
I20260807 18:37:52.133265 3449 sys_catalog.cc:458] T 00000000000000000000000000000000 P ac07d9da9d3f460caecd1af62b86a26f [sys.catalog]: This master's current role is: LEADER
I20260807 18:37:52.134955 3463 sys_catalog.cc:455] T 00000000000000000000000000000000 P 72de1af692c14c10afeee0316015bc78 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ac07d9da9d3f460caecd1af62b86a26f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac07d9da9d3f460caecd1af62b86a26f" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 45853 } } peers { permanent_uuid: "72de1af692c14c10afeee0316015bc78" member_type: VOTER last_known_addr { host: "127.11.14.125" port: 37885 } } peers { permanent_uuid: "e310196022784c039b3fd3ef9f8f73ab" member_type: VOTER last_known_addr { host: "127.11.14.124" port: 35709 } } }
I20260807 18:37:52.134949 3480 sys_catalog.cc:455] T 00000000000000000000000000000000 P e310196022784c039b3fd3ef9f8f73ab [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ac07d9da9d3f460caecd1af62b86a26f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac07d9da9d3f460caecd1af62b86a26f" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 45853 } } peers { permanent_uuid: "72de1af692c14c10afeee0316015bc78" member_type: VOTER last_known_addr { host: "127.11.14.125" port: 37885 } } peers { permanent_uuid: "e310196022784c039b3fd3ef9f8f73ab" member_type: VOTER last_known_addr { host: "127.11.14.124" port: 35709 } } }
I20260807 18:37:52.135070 3463 sys_catalog.cc:458] T 00000000000000000000000000000000 P 72de1af692c14c10afeee0316015bc78 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 18:37:52.135083 3480 sys_catalog.cc:458] T 00000000000000000000000000000000 P e310196022784c039b3fd3ef9f8f73ab [sys.catalog]: This master's current role is: FOLLOWER
I20260807 18:37:52.135355 3501 mvcc.cc:205] Tried to move back new op lower bound from 7315979764153679872 to 7315979764064010240. Current Snapshot: MvccSnapshot[applied={T|T < 7315979764153679872}]
I20260807 18:37:52.136826 3503 mvcc.cc:205] Tried to move back new op lower bound from 7315979764153679872 to 7315979764064010240. Current Snapshot: MvccSnapshot[applied={T|T < 7315979764153679872}]
I20260807 18:37:52.137048 3504 mvcc.cc:205] Tried to move back new op lower bound from 7315979764153679872 to 7315979764064010240. Current Snapshot: MvccSnapshot[applied={T|T < 7315979764153679872}]
I20260807 18:37:52.139360 3486 catalog_manager.cc:1347] Generated new cluster ID: 5968923528634d84a8dea1d6974f004d
I20260807 18:37:52.139422 3486 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:37:52.169456 3486 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:37:52.170841 3486 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:37:52.186609 3486 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ac07d9da9d3f460caecd1af62b86a26f: Generated new TSK 0
I20260807 18:37:52.187594 3486 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 18:37:52.224308 3446 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:37:52.220943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52124 (local address 127.11.14.124:35709)
0807 18:37:52.221078 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0807 18:37:52.221083 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:37:52.223098 (+ 2015us) server_negotiation.cc:407] Connection header received
0807 18:37:52.223312 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:37:52.223316 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:37:52.223373 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:37:52.223431 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:37:52.224145 (+ 714us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52124: BlockingRecv error: recv got EOF from 127.0.0.1:52124 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
W20260807 18:37:52.225841 3360 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:37:52.221417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42840 (local address 127.11.14.125:37885)
0807 18:37:52.221529 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0807 18:37:52.221533 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:37:52.224822 (+ 3289us) server_negotiation.cc:407] Connection header received
0807 18:37:52.225031 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:37:52.225035 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:37:52.225087 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:37:52.225140 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:37:52.225702 (+ 562us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42840: BlockingRecv error: recv got EOF from 127.0.0.1:42840 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
W20260807 18:37:52.227241 3273 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:37:52.221859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49738 (local address 127.11.14.126:45853)
0807 18:37:52.221977 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0807 18:37:52.221982 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:37:52.226343 (+ 4361us) server_negotiation.cc:407] Connection header received
0807 18:37:52.226528 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:37:52.226532 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:37:52.226582 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:37:52.226635 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:37:52.227133 (+ 498us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49738: BlockingRecv error: recv got EOF from 127.0.0.1:49738 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
W20260807 18:37:52.232092 3360 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:37:52.230391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42842 (local address 127.11.14.125:37885)
0807 18:37:52.230517 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0807 18:37:52.230522 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:37:52.231097 (+ 575us) server_negotiation.cc:407] Connection header received
0807 18:37:52.231320 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:37:52.231324 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:37:52.231404 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:37:52.231462 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:37:52.231997 (+ 535us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42842: BlockingRecv error: recv got EOF from 127.0.0.1:42842 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
W20260807 18:37:52.240231 3360 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:37:52.235814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42856 (local address 127.11.14.125:37885)
0807 18:37:52.235924 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0807 18:37:52.235931 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:37:52.239265 (+ 3334us) server_negotiation.cc:407] Connection header received
0807 18:37:52.239456 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:37:52.239461 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:37:52.239511 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:37:52.239586 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:37:52.240099 (+ 513us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42856: BlockingRecv error: recv got EOF from 127.0.0.1:42856 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
W20260807 18:37:52.247512 3360 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:37:52.245863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42858 (local address 127.11.14.125:37885)
0807 18:37:52.245969 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0807 18:37:52.245973 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:37:52.246495 (+ 522us) server_negotiation.cc:407] Connection header received
0807 18:37:52.246695 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:37:52.246698 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:37:52.246749 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:37:52.246803 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:37:52.247388 (+ 585us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42858: BlockingRecv error: recv got EOF from 127.0.0.1:42858 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
W20260807 18:37:52.255777 3360 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:37:52.254005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42866 (local address 127.11.14.125:37885)
0807 18:37:52.254145 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0807 18:37:52.254149 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:37:52.254757 (+ 608us) server_negotiation.cc:407] Connection header received
0807 18:37:52.254981 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:37:52.254985 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:37:52.255037 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:37:52.255094 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:37:52.255668 (+ 574us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42866: BlockingRecv error: recv got EOF from 127.0.0.1:42866 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
W20260807 18:37:52.268397 3360 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:37:52.266488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42868 (local address 127.11.14.125:37885)
0807 18:37:52.266613 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0807 18:37:52.266618 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:37:52.267242 (+ 624us) server_negotiation.cc:407] Connection header received
0807 18:37:52.267497 (+ 255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:37:52.267501 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:37:52.267607 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:37:52.267671 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:37:52.268276 (+ 605us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42868: BlockingRecv error: recv got EOF from 127.0.0.1:42868 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
W20260807 18:37:52.289327 3360 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:37:52.287360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42878 (local address 127.11.14.125:37885)
0807 18:37:52.287488 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0807 18:37:52.287493 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:37:52.288156 (+ 663us) server_negotiation.cc:407] Connection header received
0807 18:37:52.288373 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:37:52.288376 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:37:52.288431 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:37:52.288487 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:37:52.289196 (+ 709us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42878: BlockingRecv error: recv got EOF from 127.0.0.1:42878 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
W20260807 18:37:52.325662 3360 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:37:52.324205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42890 (local address 127.11.14.125:37885)
0807 18:37:52.324343 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0807 18:37:52.324348 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:37:52.324706 (+ 358us) server_negotiation.cc:407] Connection header received
0807 18:37:52.324895 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:37:52.324898 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:37:52.324949 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:37:52.325045 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:37:52.325525 (+ 480us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42890: BlockingRecv error: recv got EOF from 127.0.0.1:42890 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
W20260807 18:37:52.393538 3360 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:37:52.392070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42900 (local address 127.11.14.125:37885)
0807 18:37:52.392296 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0807 18:37:52.392300 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:37:52.392409 (+ 109us) server_negotiation.cc:407] Connection header received
0807 18:37:52.392662 (+ 253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:37:52.392666 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:37:52.392725 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:37:52.392819 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:37:52.393386 (+ 567us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42900: BlockingRecv error: recv got EOF from 127.0.0.1:42900 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
W20260807 18:37:52.525168 3360 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:37:52.523847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42906 (local address 127.11.14.125:37885)
0807 18:37:52.524014 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0807 18:37:52.524019 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:37:52.524177 (+ 158us) server_negotiation.cc:407] Connection header received
0807 18:37:52.524351 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:37:52.524355 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:37:52.524414 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:37:52.524508 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:37:52.525051 (+ 543us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42906: BlockingRecv error: recv got EOF from 127.0.0.1:42906 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
W20260807 18:37:52.784956 3360 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:37:52.783745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42910 (local address 127.11.14.125:37885)
0807 18:37:52.783907 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0807 18:37:52.783911 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:37:52.784042 (+ 131us) server_negotiation.cc:407] Connection header received
0807 18:37:52.784198 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:37:52.784200 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:37:52.784247 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:37:52.784330 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:37:52.784836 (+ 506us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42910: BlockingRecv error: recv got EOF from 127.0.0.1:42910 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
I20260807 18:37:53.082402 3477 catalog_manager.cc:1259] Loaded cluster ID: 5968923528634d84a8dea1d6974f004d
I20260807 18:37:53.082499 3477 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 72de1af692c14c10afeee0316015bc78: loading cluster ID for follower catalog manager: success
I20260807 18:37:53.084267 3477 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 72de1af692c14c10afeee0316015bc78: acquiring CA information for follower catalog manager: success
I20260807 18:37:53.085440 3477 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 72de1af692c14c10afeee0316015bc78: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 18:37:53.112601 3500 catalog_manager.cc:1259] Loaded cluster ID: 5968923528634d84a8dea1d6974f004d
I20260807 18:37:53.112710 3500 catalog_manager.cc:1552] T 00000000000000000000000000000000 P e310196022784c039b3fd3ef9f8f73ab: loading cluster ID for follower catalog manager: success
I20260807 18:37:53.114413 3500 catalog_manager.cc:1574] T 00000000000000000000000000000000 P e310196022784c039b3fd3ef9f8f73ab: acquiring CA information for follower catalog manager: success
I20260807 18:37:53.115684 3500 catalog_manager.cc:1602] T 00000000000000000000000000000000 P e310196022784c039b3fd3ef9f8f73ab: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 18:37:53.303392 3584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:53.299479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42924 (local address 127.11.14.125:37885)
0807 18:37:53.299793 (+ 314us) server_negotiation.cc:206] Beginning negotiation
0807 18:37:53.299797 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:37:53.299814 (+ 17us) server_negotiation.cc:407] Connection header received
0807 18:37:53.299870 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:37:53.299872 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:37:53.300003 (+ 131us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:37:53.300131 (+ 128us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:37:53.300405 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:53.300959 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:53.301931 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:53.302176 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:53.302261 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:53.302584 (+ 323us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:37:53.303050 (+ 466us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:37:53.303144 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0807 18:37:53.303214 (+ 70us) server_negotiation.cc:299] Negotiation successful
0807 18:37:53.303266 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":199,"thread_start_us":118,"threads_started":1}
I20260807 18:37:53.310480 3590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:53.306497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52128 (local address 127.11.14.124:35709)
0807 18:37:53.306744 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0807 18:37:53.306750 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:37:53.306803 (+ 53us) server_negotiation.cc:407] Connection header received
0807 18:37:53.306974 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:37:53.306976 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:37:53.307090 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:37:53.307218 (+ 128us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:37:53.307448 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:53.308070 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:53.308905 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:53.309124 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:53.309249 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:53.309598 (+ 349us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:37:53.310092 (+ 494us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:37:53.310229 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0807 18:37:53.310266 (+ 37us) server_negotiation.cc:299] Negotiation successful
0807 18:37:53.310322 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":169,"thread_start_us":64,"threads_started":1}
I20260807 18:37:53.317493 3584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:53.313387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42936 (local address 127.11.14.125:37885)
0807 18:37:53.313850 (+ 463us) server_negotiation.cc:206] Beginning negotiation
0807 18:37:53.313854 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:37:53.313869 (+ 15us) server_negotiation.cc:407] Connection header received
0807 18:37:53.313922 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:37:53.313927 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:37:53.313986 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:37:53.314072 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:37:53.314334 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:53.314972 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:53.316366 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:53.316583 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:53.316691 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:53.316962 (+ 271us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:37:53.317028 (+ 66us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:37:53.317108 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0807 18:37:53.317277 (+ 169us) server_negotiation.cc:299] Negotiation successful
0807 18:37:53.317337 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":361}
I20260807 18:37:53.317734 3590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:53.314130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52132 (local address 127.11.14.124:35709)
0807 18:37:53.314274 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0807 18:37:53.314279 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:37:53.314294 (+ 15us) server_negotiation.cc:407] Connection header received
0807 18:37:53.314351 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:37:53.314353 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:37:53.314388 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:37:53.314470 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:37:53.314718 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:53.315375 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:53.316324 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:53.316647 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:53.316742 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:53.317123 (+ 381us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:37:53.317205 (+ 82us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:37:53.317272 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0807 18:37:53.317566 (+ 294us) server_negotiation.cc:299] Negotiation successful
0807 18:37:53.317620 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260807 18:37:53.319171 3597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:53.313557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49746 (local address 127.11.14.126:45853)
0807 18:37:53.313798 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0807 18:37:53.313803 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:37:53.314215 (+ 412us) server_negotiation.cc:407] Connection header received
0807 18:37:53.314744 (+ 529us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:37:53.314748 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:37:53.314921 (+ 173us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:37:53.315067 (+ 146us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:37:53.315314 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:53.316151 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:53.317342 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:53.317800 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:53.317897 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:53.318305 (+ 408us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:37:53.318787 (+ 482us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:37:53.318862 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0807 18:37:53.318948 (+ 86us) server_negotiation.cc:299] Negotiation successful
0807 18:37:53.318994 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":64,"threads_started":1}
I20260807 18:37:53.322597 11321 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkz56XN/build/debug/bin/kudu with pid 3194
I20260807 18:37:53.333451 11321 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkz56XN/build/debug/bin/kudu with pid 3277
I20260807 18:37:53.341423 11321 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkz56XN/build/debug/bin/kudu with pid 3364
2026-08-07T18:37:53Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (3285 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786127579900523-11321-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:37:53 dist-test-slave-6k22 krb5kdc[3601](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:37:53 dist-test-slave-6k22 krb5kdc[3601](info): set up 2 sockets
Aug 07 18:37:53 dist-test-slave-6k22 krb5kdc[3601](info): commencing operation
krb5kdc: starting...
W20260807 18:37:54.459059 11321 mini_kdc.cc:121] Time spent starting KDC: real 1.094s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786127579900523-11321-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786127579900523-11321-0/krb5kdc/test-user.keytab.
Aug 07 18:37:54 dist-test-slave-6k22 krb5kdc[3601](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127874, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:37:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:37:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.14.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.14.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.14.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.126.keytab.
Entry for principal kudu/127.11.14.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.126.keytab.
I20260807 18:37:54.590232 11321 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkz56XN/build/debug/bin/kudu
/tmp/dist-test-taskkz56XN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786127579900523-11321-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786127579900523-11321-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786127579900523-11321-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786127579900523-11321-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.14.126:39009
--webserver_interface=127.11.14.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.126.keytab
--principal=kudu/127.11.14.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.14.84:39025
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.14.126:39009,127.11.14.125:37509,127.11.14.124:42153
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786127579900523-11321-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786127579900523-11321-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786127579900523-11321-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:37:54.704816 3615 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 18:37:54.705152 3615 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:37:54.705201 3615 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:37:54.705305 3615 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:37:54.709297 3615 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:37:54.709400 3615 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:37:54.709437 3615 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:37:54.709477 3615 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:37:54.709514 3615 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:37:54.715026 3615 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.14.84:39025
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786127579900523-11321-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786127579900523-11321-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.14.126:39009,127.11.14.125:37509,127.11.14.124:42153
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.126.keytab
--principal=kudu/127.11.14.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.14.126:39009
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786127579900523-11321-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.14.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3615
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786127579900523-11321-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:37:54.716157 3615 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:37:54.717196 3615 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:37:54.725805 3623 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:37:54.725857 3625 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:37:54.725857 3621 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:37:54.725931 3615 server_base.cc:1034] running on GCE node
I20260807 18:37:54.726728 3615 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:37:54.727903 3615 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:37:54.729074 3615 hybrid_clock.cc:648] HybridClock initialized: now 1786127874729042 us; error 41 us; skew 500 ppm
Aug 07 18:37:54 dist-test-slave-6k22 krb5kdc[3601](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127874, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:37:54.732132 3615 init.cc:377] Logged in from keytab as kudu/127.11.14.126@KRBTEST.COM (short username kudu)
I20260807 18:37:54.733212 3615 webserver.cc:460] Webserver started at http://127.11.14.126:37015/ using document root <none> and password file <none>
I20260807 18:37:54.733824 3615 fs_manager.cc:359] Metadata directory not provided
I20260807 18:37:54.733928 3615 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:37:54.734181 3615 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:37:54.736408 3615 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786127579900523-11321-0/minicluster-data/master-0/data/instance:
uuid: "95117623e6424e2f8b2f8beb692552fd"
format_stamp: "Formatted at 2026-08-07 18:37:54 on dist-test-slave-6k22"
server_key: "dddeca28a1e4ebabf3f5f01afa8ee611"
server_key_iv: "167457d36f0cf875f36ae1a0b7d57fef"
server_key_version: "encryptionkey@0"
I20260807 18:37:54.737008 3615 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786127579900523-11321-0/minicluster-data/master-0/wal/instance:
uuid: "95117623e6424e2f8b2f8beb692552fd"
format_stamp: "Formatted at 2026-08-07 18:37:54 on dist-test-slave-6k22"
server_key: "dddeca28a1e4ebabf3f5f01afa8ee611"
server_key_iv: "167457d36f0cf875f36ae1a0b7d57fef"
server_key_version: "encryptionkey@0"
I20260807 18:37:54.741269 3615 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 18:37:54.744176 3633 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:37:54.745595 3615 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:37:54.745791 3615 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786127579900523-11321-0/minicluster-data/master-0/data,/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786127579900523-11321-0/minicluster-data/master-0/wal
uuid: "95117623e6424e2f8b2f8beb692552fd"
format_stamp: "Formatted at 2026-08-07 18:37:54 on dist-test-slave-6k22"
server_key: "dddeca28a1e4ebabf3f5f01afa8ee611"
server_key_iv: "167457d36f0cf875f36ae1a0b7d57fef"
server_key_version: "encryptionkey@0"
I20260807 18:37:54.745916 3615 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786127579900523-11321-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786127579900523-11321-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786127579900523-11321-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:37:54.762064 3615 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:37:54.765302 3615 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:37:54.765533 3615 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:37:54.773448 3615 rpc_server.cc:307] RPC server started. Bound to: 127.11.14.126:39009
I20260807 18:37:54.773804 3690 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.14.126:39009 every 8 connection(s)
I20260807 18:37:54.774555 3615 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786127579900523-11321-0/minicluster-data/master-0/data/info.pb
I20260807 18:37:54.777375 11321 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkz56XN/build/debug/bin/kudu as pid 3615
I20260807 18:37:54.777535 11321 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786127579900523-11321-0/minicluster-data/master-0/wal/instance
I20260807 18:37:54.777743 3691 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:37:54.777881 11321 external_mini_cluster.cc:1428] Setting key f7f4e0028bcec181d9dfda30d0a4cc3b
I20260807 18:37:54.782119 3691 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.14.126" port: 39009 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.11.14.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.14.125@KRBTEST.COM" created.
Aug 07 18:37:54 dist-test-slave-6k22 krb5kdc[3601](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127874, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.126@KRBTEST.COM for kudu/127.11.14.126@KRBTEST.COM
I20260807 18:37:54.799311 3695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:54.784564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53978 (local address 127.11.14.126:39009)
0807 18:37:54.785173 (+ 609us) server_negotiation.cc:206] Beginning negotiation
0807 18:37:54.785186 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 18:37:54.785242 (+ 56us) server_negotiation.cc:407] Connection header received
0807 18:37:54.786279 (+ 1037us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:37:54.786289 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:37:54.786593 (+ 304us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:37:54.786833 (+ 240us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:37:54.787659 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:54.788871 (+ 1212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:54.790288 (+ 1417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:54.790683 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:54.790867 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:54.793686 (+ 2819us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:37:54.793695 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:37:54.793712 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:37:54.793752 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:37:54.795749 (+ 1997us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:37:54.796389 (+ 640us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:37:54.796394 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:37:54.796395 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:37:54.796501 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:37:54.796973 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:37:54.796976 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:37:54.796977 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:37:54.797352 (+ 375us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:37:54.797459 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0807 18:37:54.798096 (+ 637us) server_negotiation.cc:299] Negotiation successful
0807 18:37:54.798387 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":479,"thread_start_us":176,"threads_started":1}
I20260807 18:37:54.799316 3694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:54.784563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:39009 (local address 127.0.0.1:53978)
0807 18:37:54.785050 (+ 487us) negotiation.cc:107] Waiting for socket to connect
0807 18:37:54.785101 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0807 18:37:54.785932 (+ 831us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:37:54.786853 (+ 921us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:37:54.786857 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:37:54.787174 (+ 317us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:37:54.787454 (+ 280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:37:54.787467 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:37:54.789071 (+ 1604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:37:54.789078 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:37:54.790070 (+ 992us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:37:54.790082 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:37:54.790897 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:37:54.790900 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:37:54.791032 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:54.791673 (+ 641us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:37:54.791699 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:37:54.793488 (+ 1789us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:37:54.795878 (+ 2390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:37:54.795892 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:37:54.795907 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:37:54.796271 (+ 364us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:37:54.796610 (+ 339us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:37:54.796617 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:37:54.796623 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:37:54.796861 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:37:54.797493 (+ 632us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:37:54.797505 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:37:54.797745 (+ 240us) client_negotiation.cc:769] Sending connection context
0807 18:37:54.798020 (+ 275us) client_negotiation.cc:240] Negotiation successful
0807 18:37:54.798485 (+ 465us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":280,"mutex_wait_us":41,"thread_start_us":143,"threads_started":1}
I20260807 18:37:54.801961 3691 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.14.125" port: 37509 } has no permanent_uuid. Determining permanent_uuid...
I20260807 18:37:54.802639 3694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:54.802396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.125:37509 (local address 127.0.0.1:52856)
0807 18:37:54.802513 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0807 18:37:54.802585 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.125:37509: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260807 18:37:54.802848 3637 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.11.14.125:37509: connect: Connection refused (error 111)
I20260807 18:37:54.803848 3694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:54.803536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.125:37509 (local address 127.0.0.1:52872)
0807 18:37:54.803748 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0807 18:37:54.803797 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.125:37509: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":148}
W20260807 18:37:54.804039 3691 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.14.125:37509: Network error: Client connection negotiation failed: client connection to 127.11.14.125:37509: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.14.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.125.keytab.
Entry for principal kudu/127.11.14.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.125.keytab.
I20260807 18:37:54.811108 11321 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkz56XN/build/debug/bin/kudu
/tmp/dist-test-taskkz56XN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786127579900523-11321-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786127579900523-11321-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786127579900523-11321-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786127579900523-11321-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.14.125:37509
--webserver_interface=127.11.14.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.125.keytab
--principal=kudu/127.11.14.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.14.84:39025
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.14.126:39009,127.11.14.125:37509,127.11.14.124:42153
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786127579900523-11321-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786127579900523-11321-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786127579900523-11321-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 18:37:54.853322 3691 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.14.125" port: 37509 } attempt: 1
I20260807 18:37:54.854393 3694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:54.854049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.125:37509 (local address 127.0.0.1:52876)
0807 18:37:54.854227 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0807 18:37:54.854304 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.125:37509: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260807 18:37:54.855276 3694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:54.855040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.125:37509 (local address 127.0.0.1:52878)
0807 18:37:54.855158 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0807 18:37:54.855218 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.125:37509: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260807 18:37:54.855405 3691 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.14.125:37509: Network error: Client connection negotiation failed: client connection to 127.11.14.125:37509: connect: Connection refused (error 111)
I20260807 18:37:54.923637 3691 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.14.125" port: 37509 } attempt: 2
I20260807 18:37:54.924557 3694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:54.924260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.125:37509 (local address 127.0.0.1:52886)
0807 18:37:54.924417 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0807 18:37:54.924480 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.125:37509: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260807 18:37:54.925324 3694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:54.925098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.125:37509 (local address 127.0.0.1:52892)
0807 18:37:54.925214 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0807 18:37:54.925268 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.125:37509: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260807 18:37:54.925447 3691 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.14.125:37509: Network error: Client connection negotiation failed: client connection to 127.11.14.125:37509: connect: Connection refused (error 111)
W20260807 18:37:54.933055 3698 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 18:37:54.933408 3698 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:37:54.933475 3698 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:37:54.933561 3698 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:37:54.937736 3698 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:37:54.937824 3698 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:37:54.937863 3698 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:37:54.937922 3698 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:37:54.937961 3698 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:37:54.943464 3698 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.14.84:39025
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786127579900523-11321-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786127579900523-11321-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.11.14.126:39009,127.11.14.125:37509,127.11.14.124:42153
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.125.keytab
--principal=kudu/127.11.14.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.14.125:37509
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786127579900523-11321-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.11.14.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3698
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786127579900523-11321-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:37:54.944723 3698 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:37:54.945813 3698 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:37:54.954640 3703 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:37:54.954707 3706 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:37:54.954640 3704 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:37:54.955639 3698 server_base.cc:1034] running on GCE node
I20260807 18:37:54.956251 3698 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:37:54.957367 3698 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:37:54.958575 3698 hybrid_clock.cc:648] HybridClock initialized: now 1786127874958557 us; error 29 us; skew 500 ppm
Aug 07 18:37:54 dist-test-slave-6k22 krb5kdc[3601](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127874, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:37:54.961562 3698 init.cc:377] Logged in from keytab as kudu/127.11.14.125@KRBTEST.COM (short username kudu)
I20260807 18:37:54.962462 3698 webserver.cc:460] Webserver started at http://127.11.14.125:39101/ using document root <none> and password file <none>
I20260807 18:37:54.962990 3698 fs_manager.cc:359] Metadata directory not provided
I20260807 18:37:54.963050 3698 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:37:54.963231 3698 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:37:54.965335 3698 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786127579900523-11321-0/minicluster-data/master-1/data/instance:
uuid: "24a9bb5a15c64e1e9a0b98a4e019a033"
format_stamp: "Formatted at 2026-08-07 18:37:54 on dist-test-slave-6k22"
server_key: "6bf351ab772909d3896a0e966cb514ec"
server_key_iv: "809b4d99ffbb8b2889a938fdaa67c9b7"
server_key_version: "encryptionkey@0"
I20260807 18:37:54.965854 3698 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786127579900523-11321-0/minicluster-data/master-1/wal/instance:
uuid: "24a9bb5a15c64e1e9a0b98a4e019a033"
format_stamp: "Formatted at 2026-08-07 18:37:54 on dist-test-slave-6k22"
server_key: "6bf351ab772909d3896a0e966cb514ec"
server_key_iv: "809b4d99ffbb8b2889a938fdaa67c9b7"
server_key_version: "encryptionkey@0"
I20260807 18:37:54.969677 3698 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 18:37:54.972218 3714 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:37:54.973495 3698 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 18:37:54.973601 3698 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786127579900523-11321-0/minicluster-data/master-1/data,/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786127579900523-11321-0/minicluster-data/master-1/wal
uuid: "24a9bb5a15c64e1e9a0b98a4e019a033"
format_stamp: "Formatted at 2026-08-07 18:37:54 on dist-test-slave-6k22"
server_key: "6bf351ab772909d3896a0e966cb514ec"
server_key_iv: "809b4d99ffbb8b2889a938fdaa67c9b7"
server_key_version: "encryptionkey@0"
I20260807 18:37:54.973686 3698 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786127579900523-11321-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786127579900523-11321-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786127579900523-11321-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:37:54.997555 3698 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:37:55.001184 3698 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:37:55.001350 3698 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:37:55.008574 3698 rpc_server.cc:307] RPC server started. Bound to: 127.11.14.125:37509
I20260807 18:37:55.008900 3771 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.14.125:37509 every 8 connection(s)
I20260807 18:37:55.009581 3698 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786127579900523-11321-0/minicluster-data/master-1/data/info.pb
I20260807 18:37:55.012467 3773 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:37:55.016264 3773 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.14.126" port: 39009 } has no permanent_uuid. Determining permanent_uuid...
I20260807 18:37:55.016624 3691 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.14.125" port: 37509 } attempt: 3
I20260807 18:37:55.019860 11321 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkz56XN/build/debug/bin/kudu as pid 3698
I20260807 18:37:55.020007 11321 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786127579900523-11321-0/minicluster-data/master-1/wal/instance
I20260807 18:37:55.020313 11321 external_mini_cluster.cc:1428] Setting key 41d97b815d0323f9a34024bc469f3ec6
Aug 07 18:37:55 dist-test-slave-6k22 krb5kdc[3601](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127874, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.126@KRBTEST.COM for kudu/127.11.14.125@KRBTEST.COM
Aug 07 18:37:55 dist-test-slave-6k22 krb5kdc[3601](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127874, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.125@KRBTEST.COM for kudu/127.11.14.126@KRBTEST.COM
I20260807 18:37:55.030428 3694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:55.017129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.125:37509 (local address 127.0.0.1:52906)
0807 18:37:55.017272 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0807 18:37:55.017289 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 18:37:55.017390 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:37:55.019717 (+ 2327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:37:55.019720 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:37:55.019921 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:37:55.020036 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:37:55.020043 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:37:55.021786 (+ 1743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:37:55.021790 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:37:55.022374 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:37:55.022387 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:37:55.023203 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:37:55.023207 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:37:55.023381 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:55.023986 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:37:55.024005 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:37:55.025687 (+ 1682us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:37:55.028324 (+ 2637us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:37:55.028329 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:37:55.028333 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:37:55.028602 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:37:55.029026 (+ 424us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:37:55.029029 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:37:55.029031 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:37:55.029135 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:37:55.029849 (+ 714us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:37:55.029852 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:37:55.029978 (+ 126us) client_negotiation.cc:769] Sending connection context
0807 18:37:55.030093 (+ 115us) client_negotiation.cc:240] Negotiation successful
0807 18:37:55.030243 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":58,"mutex_wait_us":25,"spinlock_wait_cycles":2304}
WARNING: no policy specified for kudu/127.11.14.124@KRBTEST.COM; defaulting to no policy
I20260807 18:37:55.031453 3695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:55.018572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53984 (local address 127.11.14.126:39009)
0807 18:37:55.018691 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0807 18:37:55.018696 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:37:55.018994 (+ 298us) server_negotiation.cc:407] Connection header received
0807 18:37:55.019400 (+ 406us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:37:55.019403 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:37:55.019448 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:37:55.019547 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:37:55.020670 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:55.021426 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:55.022961 (+ 1535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:55.023441 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:55.023964 (+ 523us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:55.026825 (+ 2861us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:37:55.026831 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:37:55.026837 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:37:55.026871 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:37:55.029300 (+ 2429us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:37:55.029901 (+ 601us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:37:55.029905 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:37:55.029907 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:37:55.030016 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:37:55.030385 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:37:55.030389 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:37:55.030391 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:37:55.030590 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:37:55.030671 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0807 18:37:55.031190 (+ 519us) server_negotiation.cc:299] Negotiation successful
0807 18:37:55.031318 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":36,"server-negotiator.queue_time_us":53,"spinlock_wait_cycles":3968}
I20260807 18:37:55.031682 3776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:55.018508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:39009 (local address 127.0.0.1:53984)
0807 18:37:55.018844 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0807 18:37:55.018894 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0807 18:37:55.019188 (+ 294us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:37:55.019661 (+ 473us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:37:55.019670 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:37:55.020153 (+ 483us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:37:55.020500 (+ 347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:37:55.020517 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:37:55.021565 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:37:55.021571 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:37:55.022818 (+ 1247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:37:55.022827 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:37:55.023647 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:37:55.023651 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:37:55.023741 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:55.024244 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:37:55.024266 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:37:55.026646 (+ 2380us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:37:55.029418 (+ 2772us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:37:55.029429 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:37:55.029436 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:37:55.029759 (+ 323us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:37:55.030162 (+ 403us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:37:55.030167 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:37:55.030171 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:37:55.030266 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:37:55.030788 (+ 522us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:37:55.030798 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:37:55.030933 (+ 135us) client_negotiation.cc:769] Sending connection context
0807 18:37:55.031097 (+ 164us) client_negotiation.cc:240] Negotiation successful
0807 18:37:55.031241 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171,"mutex_wait_us":47,"thread_start_us":90,"threads_started":1}
I20260807 18:37:55.031687 3775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:55.017841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52906 (local address 127.11.14.125:37509)
0807 18:37:55.018261 (+ 420us) server_negotiation.cc:206] Beginning negotiation
0807 18:37:55.018276 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 18:37:55.018309 (+ 33us) server_negotiation.cc:407] Connection header received
0807 18:37:55.019177 (+ 868us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:37:55.019198 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:37:55.019452 (+ 254us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:37:55.019676 (+ 224us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:37:55.020216 (+ 540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:55.021652 (+ 1436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:55.022539 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:55.023069 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:55.023243 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:55.025872 (+ 2629us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:37:55.025897 (+ 25us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:37:55.025914 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:37:55.025944 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:37:55.028183 (+ 2239us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:37:55.028730 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:37:55.028738 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:37:55.028747 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:37:55.028904 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:37:55.029247 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:37:55.029252 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:37:55.029254 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:37:55.029718 (+ 464us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:37:55.029872 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0807 18:37:55.030254 (+ 382us) server_negotiation.cc:299] Negotiation successful
0807 18:37:55.030580 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":278,"thread_start_us":109,"threads_started":1}
I20260807 18:37:55.033416 3773 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.14.125" port: 37509 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.14.124@KRBTEST.COM" created.
I20260807 18:37:55.034297 3691 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.14.124" port: 42153 } has no permanent_uuid. Determining permanent_uuid...
I20260807 18:37:55.034971 3694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:55.034719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.124:42153 (local address 127.0.0.1:46200)
0807 18:37:55.034837 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0807 18:37:55.034885 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.124:42153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260807 18:37:55.035924 3694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:55.035432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.124:42153 (local address 127.0.0.1:46216)
0807 18:37:55.035760 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0807 18:37:55.035825 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.124:42153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":144,"threads_started":1}
W20260807 18:37:55.036159 3691 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.14.124:42153: Network error: Client connection negotiation failed: client connection to 127.11.14.124:42153: connect: Connection refused (error 111)
Aug 07 18:37:55 dist-test-slave-6k22 krb5kdc[3601](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127874, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.125@KRBTEST.COM for kudu/127.11.14.125@KRBTEST.COM
I20260807 18:37:55.042263 3775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:55.034013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52920 (local address 127.11.14.125:37509)
0807 18:37:55.034273 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0807 18:37:55.034282 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:37:55.034292 (+ 10us) server_negotiation.cc:407] Connection header received
0807 18:37:55.034316 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:37:55.034318 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:37:55.034347 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:37:55.034429 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:37:55.034961 (+ 532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:55.035743 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:55.036667 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:55.036918 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:55.037028 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:55.039177 (+ 2149us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:37:55.039181 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:37:55.039183 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:37:55.039205 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:37:55.040673 (+ 1468us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:37:55.041097 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:37:55.041102 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:37:55.041104 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:37:55.041184 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:37:55.041458 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:37:55.041461 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:37:55.041463 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:37:55.041631 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:37:55.041736 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0807 18:37:55.041976 (+ 240us) server_negotiation.cc:299] Negotiation successful
0807 18:37:55.042129 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":57,"server-negotiator.queue_time_us":180}
I20260807 18:37:55.042263 3776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:55.033951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.125:37509 (local address 127.0.0.1:52920)
0807 18:37:55.034090 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0807 18:37:55.034104 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 18:37:55.034188 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:37:55.034440 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:37:55.034443 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:37:55.034635 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:37:55.034831 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:37:55.034841 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:37:55.035869 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:37:55.035872 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:37:55.036549 (+ 677us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:37:55.036559 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:37:55.037028 (+ 469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:37:55.037032 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:37:55.037119 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:55.037578 (+ 459us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:37:55.037589 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:37:55.039045 (+ 1456us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:37:55.040809 (+ 1764us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:37:55.040812 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:37:55.040814 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:37:55.041000 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:37:55.041278 (+ 278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:37:55.041281 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:37:55.041283 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:37:55.041364 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:37:55.041726 (+ 362us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:37:55.041729 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:37:55.041825 (+ 96us) client_negotiation.cc:769] Sending connection context
0807 18:37:55.041932 (+ 107us) client_negotiation.cc:240] Negotiation successful
0807 18:37:55.042071 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":60}
I20260807 18:37:55.043054 3773 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.14.124" port: 42153 } has no permanent_uuid. Determining permanent_uuid...
I20260807 18:37:55.043743 3776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:55.043457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.124:42153 (local address 127.0.0.1:46222)
0807 18:37:55.043599 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0807 18:37:55.043686 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.124:42153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260807 18:37:55.044427 3715 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.11.14.124:42153: connect: Connection refused (error 111)
I20260807 18:37:55.045293 3776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:55.045066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.124:42153 (local address 127.0.0.1:46236)
0807 18:37:55.045176 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0807 18:37:55.045230 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.124:42153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260807 18:37:55.045465 3773 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.14.124:42153: Network error: Client connection negotiation failed: client connection to 127.11.14.124:42153: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.14.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.124.keytab.
Entry for principal kudu/127.11.14.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.124.keytab.
I20260807 18:37:55.052438 11321 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskkz56XN/build/debug/bin/kudu
/tmp/dist-test-taskkz56XN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786127579900523-11321-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786127579900523-11321-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786127579900523-11321-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786127579900523-11321-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.14.124:42153
--webserver_interface=127.11.14.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.124.keytab
--principal=kudu/127.11.14.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.14.84:39025
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.14.126:39009,127.11.14.125:37509,127.11.14.124:42153
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786127579900523-11321-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786127579900523-11321-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786127579900523-11321-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 18:37:55.067358 3691 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.14.124" port: 42153 } attempt: 1
I20260807 18:37:55.068368 3694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:55.067984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.124:42153 (local address 127.0.0.1:46252)
0807 18:37:55.068179 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0807 18:37:55.068261 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.124:42153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260807 18:37:55.069165 3694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:55.068913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.124:42153 (local address 127.0.0.1:46256)
0807 18:37:55.069030 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0807 18:37:55.069089 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.124:42153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260807 18:37:55.069371 3691 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.14.124:42153: Network error: Client connection negotiation failed: client connection to 127.11.14.124:42153: connect: Connection refused (error 111)
I20260807 18:37:55.094717 3773 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.14.124" port: 42153 } attempt: 1
I20260807 18:37:55.095731 3776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:55.095375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.124:42153 (local address 127.0.0.1:46264)
0807 18:37:55.095540 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0807 18:37:55.095651 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.124:42153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260807 18:37:55.096592 3776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:55.096382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.124:42153 (local address 127.0.0.1:46266)
0807 18:37:55.096488 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0807 18:37:55.096535 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.124:42153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260807 18:37:55.096769 3773 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.14.124:42153: Network error: Client connection negotiation failed: client connection to 127.11.14.124:42153: connect: Connection refused (error 111)
I20260807 18:37:55.144587 3691 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.14.124" port: 42153 } attempt: 2
I20260807 18:37:55.145561 3694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:55.145183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.124:42153 (local address 127.0.0.1:46282)
0807 18:37:55.145376 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0807 18:37:55.145454 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.124:42153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260807 18:37:55.146360 3694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:55.146097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.124:42153 (local address 127.0.0.1:46288)
0807 18:37:55.146222 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0807 18:37:55.146287 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.124:42153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260807 18:37:55.146526 3691 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.14.124:42153: Network error: Client connection negotiation failed: client connection to 127.11.14.124:42153: connect: Connection refused (error 111)
I20260807 18:37:55.164960 3773 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.14.124" port: 42153 } attempt: 2
I20260807 18:37:55.165854 3776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:55.165535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.124:42153 (local address 127.0.0.1:46302)
0807 18:37:55.165686 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0807 18:37:55.165750 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.124:42153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260807 18:37:55.166592 3776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:55.166376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.124:42153 (local address 127.0.0.1:46306)
0807 18:37:55.166486 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0807 18:37:55.166537 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.14.124:42153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260807 18:37:55.166713 3773 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.14.124:42153: Network error: Client connection negotiation failed: client connection to 127.11.14.124:42153: connect: Connection refused (error 111)
W20260807 18:37:55.175066 3781 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 18:37:55.175410 3781 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:37:55.175477 3781 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:37:55.175612 3781 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:37:55.180315 3781 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:37:55.180398 3781 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:37:55.180449 3781 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:37:55.180491 3781 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:37:55.180534 3781 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:37:55.185807 3781 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.14.84:39025
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786127579900523-11321-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786127579900523-11321-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.11.14.126:39009,127.11.14.125:37509,127.11.14.124:42153
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786127579900523-11321-0/krb5kdc/kudu_127.11.14.124.keytab
--principal=kudu/127.11.14.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.14.124:42153
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786127579900523-11321-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.11.14.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3781
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786127579900523-11321-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 17:57:58 UTC on ccycloud.quasar-thtedw.root.comops.site
build id 4573428
I20260807 18:37:55.186906 3781 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:37:55.188081 3781 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:37:55.196707 3786 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:37:55.196707 3789 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:37:55.196903 3781 server_base.cc:1034] running on GCE node
W20260807 18:37:55.196739 3787 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:37:55.197630 3781 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:37:55.198798 3781 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:37:55.199992 3781 hybrid_clock.cc:648] HybridClock initialized: now 1786127875199978 us; error 25 us; skew 500 ppm
Aug 07 18:37:55 dist-test-slave-6k22 krb5kdc[3601](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127875, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:37:55.202788 3781 init.cc:377] Logged in from keytab as kudu/127.11.14.124@KRBTEST.COM (short username kudu)
I20260807 18:37:55.203755 3781 webserver.cc:460] Webserver started at http://127.11.14.124:44515/ using document root <none> and password file <none>
I20260807 18:37:55.204272 3781 fs_manager.cc:359] Metadata directory not provided
I20260807 18:37:55.204331 3781 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:37:55.204496 3781 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:37:55.206462 3781 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786127579900523-11321-0/minicluster-data/master-2/data/instance:
uuid: "f2e4bd71d50e45dc85785f371d890706"
format_stamp: "Formatted at 2026-08-07 18:37:55 on dist-test-slave-6k22"
server_key: "10ef6bd715ddfdcca6e0d30641d833c0"
server_key_iv: "809bb34e6c1147496b6bbf020a511338"
server_key_version: "encryptionkey@0"
I20260807 18:37:55.206990 3781 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786127579900523-11321-0/minicluster-data/master-2/wal/instance:
uuid: "f2e4bd71d50e45dc85785f371d890706"
format_stamp: "Formatted at 2026-08-07 18:37:55 on dist-test-slave-6k22"
server_key: "10ef6bd715ddfdcca6e0d30641d833c0"
server_key_iv: "809bb34e6c1147496b6bbf020a511338"
server_key_version: "encryptionkey@0"
I20260807 18:37:55.210696 3781 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.003s
I20260807 18:37:55.213244 3797 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:37:55.214574 3781 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 18:37:55.214677 3781 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786127579900523-11321-0/minicluster-data/master-2/data,/tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786127579900523-11321-0/minicluster-data/master-2/wal
uuid: "f2e4bd71d50e45dc85785f371d890706"
format_stamp: "Formatted at 2026-08-07 18:37:55 on dist-test-slave-6k22"
server_key: "10ef6bd715ddfdcca6e0d30641d833c0"
server_key_iv: "809bb34e6c1147496b6bbf020a511338"
server_key_version: "encryptionkey@0"
I20260807 18:37:55.214803 3781 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786127579900523-11321-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786127579900523-11321-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786127579900523-11321-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:37:55.226802 3781 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:37:55.229954 3781 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:37:55.230105 3781 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:37:55.237239 3781 rpc_server.cc:307] RPC server started. Bound to: 127.11.14.124:42153
I20260807 18:37:55.237509 3858 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.14.124:42153 every 8 connection(s)
I20260807 18:37:55.238237 3781 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786127579900523-11321-0/minicluster-data/master-2/data/info.pb
I20260807 18:37:55.239454 11321 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskkz56XN/build/debug/bin/kudu as pid 3781
I20260807 18:37:55.239620 11321 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskkz56XN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786127579900523-11321-0/minicluster-data/master-2/wal/instance
I20260807 18:37:55.239925 11321 external_mini_cluster.cc:1428] Setting key 3ac541fd3ff7d7e68ccaf92c6bf219ea
I20260807 18:37:55.241632 3859 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:37:55.245711 3691 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.14.124" port: 42153 } attempt: 3
I20260807 18:37:55.246006 3859 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.14.126" port: 39009 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 18:37:55 dist-test-slave-6k22 krb5kdc[3601](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127874, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.14.124@KRBTEST.COM
Aug 07 18:37:55 dist-test-slave-6k22 krb5kdc[3601](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127874, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.126@KRBTEST.COM for kudu/127.11.14.124@KRBTEST.COM
Aug 07 18:37:55 dist-test-slave-6k22 krb5kdc[3601](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127875, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.124@KRBTEST.COM for kudu/127.11.14.126@KRBTEST.COM
I20260807 18:37:55.256099 3862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:55.241260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46320 (local address 127.11.14.124:42153)
0807 18:37:55.241660 (+ 400us) server_negotiation.cc:206] Beginning negotiation
0807 18:37:55.241684 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0807 18:37:55.241732 (+ 48us) server_negotiation.cc:407] Connection header received
0807 18:37:55.242411 (+ 679us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:37:55.242428 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:37:55.242753 (+ 325us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:37:55.243029 (+ 276us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:37:55.243993 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:55.245000 (+ 1007us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:55.246216 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:55.246895 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:55.247046 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:55.250366 (+ 3320us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:37:55.250372 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:37:55.250384 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:37:55.250419 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:37:55.252548 (+ 2129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:37:55.253041 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:37:55.253047 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:37:55.253053 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:37:55.253222 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:37:55.253557 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:37:55.253560 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:37:55.253562 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:37:55.254095 (+ 533us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:37:55.254314 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0807 18:37:55.254641 (+ 327us) server_negotiation.cc:299] Negotiation successful
0807 18:37:55.254975 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":263,"thread_start_us":145,"threads_started":1}
I20260807 18:37:55.256987 3694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:55.246241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.124:42153 (local address 127.0.0.1:46328)
0807 18:37:55.246478 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0807 18:37:55.246495 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 18:37:55.246586 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:37:55.247295 (+ 709us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:37:55.247298 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:37:55.247496 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:37:55.247992 (+ 496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:37:55.248006 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:37:55.249203 (+ 1197us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:37:55.249207 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:37:55.249878 (+ 671us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:37:55.249888 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:37:55.250488 (+ 600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:37:55.250492 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:37:55.250592 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:55.251153 (+ 561us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:37:55.251174 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:37:55.252681 (+ 1507us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:37:55.254772 (+ 2091us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:37:55.254775 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:37:55.254777 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:37:55.255030 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:37:55.255310 (+ 280us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:37:55.255313 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:37:55.255315 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:37:55.255407 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:37:55.256359 (+ 952us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:37:55.256362 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:37:55.256469 (+ 107us) client_negotiation.cc:769] Sending connection context
0807 18:37:55.256555 (+ 86us) client_negotiation.cc:240] Negotiation successful
0807 18:37:55.256781 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"mutex_wait_us":24}
I20260807 18:37:55.257107 3863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:55.246302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46328 (local address 127.11.14.124:42153)
0807 18:37:55.246809 (+ 507us) server_negotiation.cc:206] Beginning negotiation
0807 18:37:55.246815 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:37:55.246825 (+ 10us) server_negotiation.cc:407] Connection header received
0807 18:37:55.246852 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:37:55.246853 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:37:55.246886 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:37:55.246989 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:37:55.248150 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:55.249085 (+ 935us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:55.250004 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:55.250371 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:55.250480 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:55.252823 (+ 2343us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:37:55.252827 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:37:55.252830 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:37:55.252849 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:37:55.254642 (+ 1793us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:37:55.255135 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:37:55.255137 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:37:55.255138 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:37:55.255205 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:37:55.255513 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:37:55.255516 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:37:55.255518 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:37:55.255998 (+ 480us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:37:55.256736 (+ 738us) server_negotiation.cc:1035] Waiting for connection context
0807 18:37:55.256806 (+ 70us) server_negotiation.cc:299] Negotiation successful
0807 18:37:55.256910 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":53,"server-negotiator.queue_time_us":451,"thread_start_us":149,"threads_started":1}
I20260807 18:37:55.257881 3773 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.14.124" port: 42153 } attempt: 3
I20260807 18:37:55.259089 3695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:55.248015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54000 (local address 127.11.14.126:39009)
0807 18:37:55.248134 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0807 18:37:55.248138 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:37:55.248491 (+ 353us) server_negotiation.cc:407] Connection header received
0807 18:37:55.248691 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:37:55.248694 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:37:55.248740 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:37:55.248864 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:37:55.249749 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:55.250385 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:55.251609 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:55.251925 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:55.252044 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:55.255388 (+ 3344us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:37:55.255391 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:37:55.255393 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:37:55.255413 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:37:55.256961 (+ 1548us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:37:55.257556 (+ 595us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:37:55.257559 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:37:55.257561 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:37:55.257642 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:37:55.257959 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:37:55.257962 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:37:55.257964 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:37:55.258247 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:37:55.258321 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0807 18:37:55.258817 (+ 496us) server_negotiation.cc:299] Negotiation successful
0807 18:37:55.258946 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":56}
I20260807 18:37:55.259117 3864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:55.247983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.126:39009 (local address 127.0.0.1:54000)
0807 18:37:55.248373 (+ 390us) negotiation.cc:107] Waiting for socket to connect
0807 18:37:55.248420 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0807 18:37:55.248571 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:37:55.248944 (+ 373us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:37:55.248950 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:37:55.249461 (+ 511us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:37:55.249618 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:37:55.249626 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:37:55.250517 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:37:55.250521 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:37:55.251471 (+ 950us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:37:55.251483 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:37:55.252048 (+ 565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:37:55.252052 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:37:55.252170 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:55.252721 (+ 551us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:37:55.252756 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:37:55.255273 (+ 2517us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:37:55.257092 (+ 1819us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:37:55.257102 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:37:55.257110 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:37:55.257443 (+ 333us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:37:55.257748 (+ 305us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:37:55.257751 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:37:55.257753 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:37:55.257847 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:37:55.258429 (+ 582us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:37:55.258435 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:37:55.258555 (+ 120us) client_negotiation.cc:769] Sending connection context
0807 18:37:55.258708 (+ 153us) client_negotiation.cc:240] Negotiation successful
0807 18:37:55.258910 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":87,"threads_started":1}
I20260807 18:37:55.260813 3859 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.14.125" port: 37509 } has no permanent_uuid. Determining permanent_uuid...
I20260807 18:37:55.262339 3691 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 95117623e6424e2f8b2f8beb692552fd: Bootstrap starting.
Aug 07 18:37:55 dist-test-slave-6k22 krb5kdc[3601](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127874, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.125@KRBTEST.COM for kudu/127.11.14.124@KRBTEST.COM
I20260807 18:37:55.265578 3691 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 95117623e6424e2f8b2f8beb692552fd: Neither blocks nor log segments found. Creating new log.
Aug 07 18:37:55 dist-test-slave-6k22 krb5kdc[3601](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127874, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.14.126@KRBTEST.COM
I20260807 18:37:55.266623 3691 log.cc:824] T 00000000000000000000000000000000 P 95117623e6424e2f8b2f8beb692552fd: Log is configured to *not* fsync() on all Append() calls
I20260807 18:37:55.268136 3776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:55.258340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.124:42153 (local address 127.0.0.1:46330)
0807 18:37:55.258833 (+ 493us) negotiation.cc:107] Waiting for socket to connect
0807 18:37:55.258848 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 18:37:55.258940 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:37:55.259361 (+ 421us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:37:55.259364 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:37:55.259544 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:37:55.259683 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:37:55.259690 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:37:55.260655 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:37:55.260658 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:37:55.261453 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:37:55.261465 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:37:55.262087 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:37:55.262091 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:37:55.262179 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:55.262676 (+ 497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:37:55.262690 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:37:55.264278 (+ 1588us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:37:55.266348 (+ 2070us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:37:55.266352 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:37:55.266354 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:37:55.266611 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:37:55.266972 (+ 361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:37:55.266978 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:37:55.266982 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:37:55.267105 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:37:55.267599 (+ 494us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:37:55.267603 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:37:55.267698 (+ 95us) client_negotiation.cc:769] Sending connection context
0807 18:37:55.267825 (+ 127us) client_negotiation.cc:240] Negotiation successful
0807 18:37:55.267964 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":387}
I20260807 18:37:55.268136 3863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:55.258331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46330 (local address 127.11.14.124:42153)
0807 18:37:55.258447 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0807 18:37:55.258451 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:37:55.259135 (+ 684us) server_negotiation.cc:407] Connection header received
0807 18:37:55.259178 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:37:55.259181 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:37:55.259248 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:37:55.259324 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:37:55.259837 (+ 513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:55.260521 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:55.261611 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:55.261944 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:55.262406 (+ 462us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:55.264430 (+ 2024us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:37:55.264434 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:37:55.264436 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:37:55.264461 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:37:55.266214 (+ 1753us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:37:55.266749 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:37:55.266752 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:37:55.266754 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:37:55.266852 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:37:55.267309 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:37:55.267311 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:37:55.267313 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:37:55.267467 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:37:55.267626 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0807 18:37:55.267895 (+ 269us) server_negotiation.cc:299] Negotiation successful
0807 18:37:55.268006 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
Aug 07 18:37:55 dist-test-slave-6k22 krb5kdc[3601](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127875, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.124@KRBTEST.COM for kudu/127.11.14.125@KRBTEST.COM
I20260807 18:37:55.270663 3695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:55.259171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54004 (local address 127.11.14.126:39009)
0807 18:37:55.260076 (+ 905us) server_negotiation.cc:206] Beginning negotiation
0807 18:37:55.260081 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:37:55.260094 (+ 13us) server_negotiation.cc:407] Connection header received
0807 18:37:55.260128 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:37:55.260131 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:37:55.260177 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:37:55.260246 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:37:55.261103 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:55.261789 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:55.262923 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:55.263242 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:55.263315 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:55.266726 (+ 3411us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:37:55.266732 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:37:55.266740 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:37:55.266777 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:37:55.268475 (+ 1698us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:37:55.268999 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:37:55.269002 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:37:55.269004 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:37:55.269090 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:37:55.269737 (+ 647us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:37:55.269740 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:37:55.269742 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:37:55.269979 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:37:55.270154 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0807 18:37:55.270367 (+ 213us) server_negotiation.cc:299] Negotiation successful
0807 18:37:55.270483 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":770,"server-negotiator.queue_time_us":822}
I20260807 18:37:55.271727 3773 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 24a9bb5a15c64e1e9a0b98a4e019a033: Bootstrap starting.
I20260807 18:37:55.272425 3864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:55.261296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.125:37509 (local address 127.0.0.1:52930)
0807 18:37:55.261726 (+ 430us) negotiation.cc:107] Waiting for socket to connect
0807 18:37:55.261739 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 18:37:55.261856 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:37:55.262189 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:37:55.262192 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:37:55.262664 (+ 472us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:37:55.262775 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:37:55.262783 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:37:55.264074 (+ 1291us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:37:55.264077 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:37:55.265548 (+ 1471us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:37:55.265563 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:37:55.266357 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:37:55.266363 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:37:55.266474 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:55.266917 (+ 443us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:37:55.266936 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:37:55.268879 (+ 1943us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:37:55.270742 (+ 1863us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:37:55.270745 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:37:55.270747 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:37:55.270995 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:37:55.271287 (+ 292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:37:55.271290 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:37:55.271292 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:37:55.271385 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:37:55.271923 (+ 538us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:37:55.271927 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:37:55.272027 (+ 100us) client_negotiation.cc:769] Sending connection context
0807 18:37:55.272133 (+ 106us) client_negotiation.cc:240] Negotiation successful
0807 18:37:55.272277 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":370}
I20260807 18:37:55.272442 3775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:55.261524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52930 (local address 127.11.14.125:37509)
0807 18:37:55.261622 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0807 18:37:55.261626 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:37:55.261792 (+ 166us) server_negotiation.cc:407] Connection header received
0807 18:37:55.261966 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:37:55.261970 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:37:55.262014 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:37:55.262061 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:37:55.262923 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:55.263663 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:55.265732 (+ 2069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:55.266089 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:55.266184 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:55.268985 (+ 2801us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:37:55.268989 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:37:55.268991 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:37:55.269015 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:37:55.270615 (+ 1600us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:37:55.271102 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:37:55.271105 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:37:55.271106 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:37:55.271185 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:37:55.271478 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:37:55.271480 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:37:55.271482 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:37:55.271828 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:37:55.271934 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0807 18:37:55.272198 (+ 264us) server_negotiation.cc:299] Negotiation successful
0807 18:37:55.272308 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260807 18:37:55.273226 3859 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.14.124" port: 42153 } has no permanent_uuid. Determining permanent_uuid...
I20260807 18:37:55.274565 3773 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 24a9bb5a15c64e1e9a0b98a4e019a033: Neither blocks nor log segments found. Creating new log.
I20260807 18:37:55.275477 3773 log.cc:824] T 00000000000000000000000000000000 P 24a9bb5a15c64e1e9a0b98a4e019a033: Log is configured to *not* fsync() on all Append() calls
I20260807 18:37:55.278146 3691 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 95117623e6424e2f8b2f8beb692552fd: No bootstrap required, opened a new log
Aug 07 18:37:55 dist-test-slave-6k22 krb5kdc[3601](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127875, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.14.124@KRBTEST.COM for kudu/127.11.14.124@KRBTEST.COM
Aug 07 18:37:55 dist-test-slave-6k22 krb5kdc[3601](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786127874, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.14.125@KRBTEST.COM
I20260807 18:37:55.282823 3691 raft_consensus.cc:348] T 00000000000000000000000000000000 P 95117623e6424e2f8b2f8beb692552fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95117623e6424e2f8b2f8beb692552fd" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 39009 } } peers { permanent_uuid: "24a9bb5a15c64e1e9a0b98a4e019a033" member_type: VOTER last_known_addr { host: "127.11.14.125" port: 37509 } } peers { permanent_uuid: "f2e4bd71d50e45dc85785f371d890706" member_type: VOTER last_known_addr { host: "127.11.14.124" port: 42153 } }
I20260807 18:37:55.283746 3864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:55.273755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.14.124:42153 (local address 127.0.0.1:46342)
0807 18:37:55.273865 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0807 18:37:55.273878 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 18:37:55.273952 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:37:55.274226 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:37:55.274229 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:37:55.274378 (+ 149us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:37:55.274448 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:37:55.274453 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:37:55.275412 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:37:55.275415 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:37:55.276034 (+ 619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:37:55.276044 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:37:55.276675 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:37:55.276681 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:37:55.276764 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:55.277294 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:37:55.277308 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:37:55.278565 (+ 1257us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:37:55.280820 (+ 2255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:37:55.280825 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:37:55.280827 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:37:55.281107 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:37:55.282586 (+ 1479us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:37:55.282590 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:37:55.282592 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:37:55.282701 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:37:55.283196 (+ 495us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:37:55.283202 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:37:55.283296 (+ 94us) client_negotiation.cc:769] Sending connection context
0807 18:37:55.283402 (+ 106us) client_negotiation.cc:240] Negotiation successful
0807 18:37:55.283535 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260807 18:37:55.284152 3863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:55.273817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46342 (local address 127.11.14.124:42153)
0807 18:37:55.273922 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0807 18:37:55.273926 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:37:55.273940 (+ 14us) server_negotiation.cc:407] Connection header received
0807 18:37:55.274016 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:37:55.274019 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:37:55.274058 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:37:55.274144 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:37:55.274575 (+ 431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:55.275291 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:55.276153 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:55.276457 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:55.276562 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:55.278707 (+ 2145us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:37:55.278711 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:37:55.278713 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:37:55.278738 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:37:55.280657 (+ 1919us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:37:55.282378 (+ 1721us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:37:55.282383 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:37:55.282384 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:37:55.282471 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:37:55.282829 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:37:55.282834 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:37:55.282839 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:37:55.283056 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:37:55.283159 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0807 18:37:55.283497 (+ 338us) server_negotiation.cc:299] Negotiation successful
0807 18:37:55.283668 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260807 18:37:55.284683 3773 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 24a9bb5a15c64e1e9a0b98a4e019a033: No bootstrap required, opened a new log
I20260807 18:37:55.286986 3775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:55.277281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52942 (local address 127.11.14.125:37509)
0807 18:37:55.277394 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0807 18:37:55.277397 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 18:37:55.277413 (+ 16us) server_negotiation.cc:407] Connection header received
0807 18:37:55.277450 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:37:55.277453 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:37:55.277491 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:37:55.277563 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:37:55.278427 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:55.279057 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:55.280135 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:55.280456 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:55.280555 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:55.283600 (+ 3045us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:37:55.283604 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:37:55.283607 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:37:55.283634 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:37:55.285338 (+ 1704us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:37:55.285761 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:37:55.285765 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:37:55.285766 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:37:55.285852 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:37:55.286132 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:37:55.286136 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:37:55.286138 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:37:55.286323 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:37:55.286506 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0807 18:37:55.286728 (+ 222us) server_negotiation.cc:299] Negotiation successful
0807 18:37:55.286844 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260807 18:37:55.287667 3691 raft_consensus.cc:729] T 00000000000000000000000000000000 P 95117623e6424e2f8b2f8beb692552fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95117623e6424e2f8b2f8beb692552fd, State: Initialized, Role: FOLLOWER
I20260807 18:37:55.287729 3859 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f2e4bd71d50e45dc85785f371d890706: Bootstrap starting.
I20260807 18:37:55.288076 3691 consensus_queue.cc:260] T 00000000000000000000000000000000 P 95117623e6424e2f8b2f8beb692552fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95117623e6424e2f8b2f8beb692552fd" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 39009 } } peers { permanent_uuid: "24a9bb5a15c64e1e9a0b98a4e019a033" member_type: VOTER last_known_addr { host: "127.11.14.125" port: 37509 } } peers { permanent_uuid: "f2e4bd71d50e45dc85785f371d890706" member_type: VOTER last_known_addr { host: "127.11.14.124" port: 42153 } }
I20260807 18:37:55.288611 3867 sys_catalog.cc:455] T 00000000000000000000000000000000 P 95117623e6424e2f8b2f8beb692552fd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95117623e6424e2f8b2f8beb692552fd" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 39009 } } peers { permanent_uuid: "24a9bb5a15c64e1e9a0b98a4e019a033" member_type: VOTER last_known_addr { host: "127.11.14.125" port: 37509 } } peers { permanent_uuid: "f2e4bd71d50e45dc85785f371d890706" member_type: VOTER last_known_addr { host: "127.11.14.124" port: 42153 } } }
I20260807 18:37:55.288848 3867 sys_catalog.cc:458] T 00000000000000000000000000000000 P 95117623e6424e2f8b2f8beb692552fd [sys.catalog]: This master's current role is: FOLLOWER
I20260807 18:37:55.288757 3773 raft_consensus.cc:348] T 00000000000000000000000000000000 P 24a9bb5a15c64e1e9a0b98a4e019a033 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95117623e6424e2f8b2f8beb692552fd" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 39009 } } peers { permanent_uuid: "24a9bb5a15c64e1e9a0b98a4e019a033" member_type: VOTER last_known_addr { host: "127.11.14.125" port: 37509 } } peers { permanent_uuid: "f2e4bd71d50e45dc85785f371d890706" member_type: VOTER last_known_addr { host: "127.11.14.124" port: 42153 } }
I20260807 18:37:55.289022 3773 raft_consensus.cc:729] T 00000000000000000000000000000000 P 24a9bb5a15c64e1e9a0b98a4e019a033 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24a9bb5a15c64e1e9a0b98a4e019a033, State: Initialized, Role: FOLLOWER
I20260807 18:37:55.289359 3773 consensus_queue.cc:260] T 00000000000000000000000000000000 P 24a9bb5a15c64e1e9a0b98a4e019a033 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95117623e6424e2f8b2f8beb692552fd" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 39009 } } peers { permanent_uuid: "24a9bb5a15c64e1e9a0b98a4e019a033" member_type: VOTER last_known_addr { host: "127.11.14.125" port: 37509 } } peers { permanent_uuid: "f2e4bd71d50e45dc85785f371d890706" member_type: VOTER last_known_addr { host: "127.11.14.124" port: 42153 } }
I20260807 18:37:55.289551 3691 sys_catalog.cc:564] T 00000000000000000000000000000000 P 95117623e6424e2f8b2f8beb692552fd [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:37:55.289842 3868 sys_catalog.cc:455] T 00000000000000000000000000000000 P 24a9bb5a15c64e1e9a0b98a4e019a033 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95117623e6424e2f8b2f8beb692552fd" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 39009 } } peers { permanent_uuid: "24a9bb5a15c64e1e9a0b98a4e019a033" member_type: VOTER last_known_addr { host: "127.11.14.125" port: 37509 } } peers { permanent_uuid: "f2e4bd71d50e45dc85785f371d890706" member_type: VOTER last_known_addr { host: "127.11.14.124" port: 42153 } } }
I20260807 18:37:55.290064 3868 sys_catalog.cc:458] T 00000000000000000000000000000000 P 24a9bb5a15c64e1e9a0b98a4e019a033 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 18:37:55.290536 3859 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f2e4bd71d50e45dc85785f371d890706: Neither blocks nor log segments found. Creating new log.
I20260807 18:37:55.290612 3773 sys_catalog.cc:564] T 00000000000000000000000000000000 P 24a9bb5a15c64e1e9a0b98a4e019a033 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:37:55.291469 3859 log.cc:824] T 00000000000000000000000000000000 P f2e4bd71d50e45dc85785f371d890706: Log is configured to *not* fsync() on all Append() calls
I20260807 18:37:55.296957 3859 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f2e4bd71d50e45dc85785f371d890706: No bootstrap required, opened a new log
I20260807 18:37:55.297097 11321 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260807 18:37:55.297113 3885 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 95117623e6424e2f8b2f8beb692552fd: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 18:37:55.297176 3885 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 18:37:55.298660 3662 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "95117623e6424e2f8b2f8beb692552fd"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54004
I20260807 18:37:55.298867 3662 raft_consensus.cc:482] T 00000000000000000000000000000000 P 95117623e6424e2f8b2f8beb692552fd [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 18:37:55.298964 3662 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 95117623e6424e2f8b2f8beb692552fd [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:37:55.300459 3662 raft_consensus.cc:504] T 00000000000000000000000000000000 P 95117623e6424e2f8b2f8beb692552fd [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95117623e6424e2f8b2f8beb692552fd" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 39009 } } peers { permanent_uuid: "24a9bb5a15c64e1e9a0b98a4e019a033" member_type: VOTER last_known_addr { host: "127.11.14.125" port: 37509 } } peers { permanent_uuid: "f2e4bd71d50e45dc85785f371d890706" member_type: VOTER last_known_addr { host: "127.11.14.124" port: 42153 } }
I20260807 18:37:55.301383 3662 leader_election.cc:288] T 00000000000000000000000000000000 P 95117623e6424e2f8b2f8beb692552fd [CANDIDATE]: Term 1 election: Requested vote from peers 24a9bb5a15c64e1e9a0b98a4e019a033 (127.11.14.125:37509), f2e4bd71d50e45dc85785f371d890706 (127.11.14.124:42153)
I20260807 18:37:55.301785 3744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "95117623e6424e2f8b2f8beb692552fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "24a9bb5a15c64e1e9a0b98a4e019a033"
I20260807 18:37:55.302003 3744 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 24a9bb5a15c64e1e9a0b98a4e019a033 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:37:55.298833 3698 master_runner.cc:186] Error getting master registration for 127.11.14.124:42153: OK, instance_id { permanent_uuid: "f2e4bd71d50e45dc85785f371d890706" instance_seqno: 1786127875235123 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 18:37:55.302793 3698 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 18:37:55.303229 3744 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 24a9bb5a15c64e1e9a0b98a4e019a033 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95117623e6424e2f8b2f8beb692552fd in term 1.
I20260807 18:37:55.303726 3637 leader_election.cc:302] T 00000000000000000000000000000000 P 95117623e6424e2f8b2f8beb692552fd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24a9bb5a15c64e1e9a0b98a4e019a033, 95117623e6424e2f8b2f8beb692552fd; no voters:
I20260807 18:37:55.304014 3867 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 95117623e6424e2f8b2f8beb692552fd [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:37:55.304214 3867 raft_consensus.cc:686] T 00000000000000000000000000000000 P 95117623e6424e2f8b2f8beb692552fd [term 1 LEADER]: Becoming Leader. State: Replica: 95117623e6424e2f8b2f8beb692552fd, State: Running, Role: LEADER
I20260807 18:37:55.304527 3867 consensus_queue.cc:237] T 00000000000000000000000000000000 P 95117623e6424e2f8b2f8beb692552fd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95117623e6424e2f8b2f8beb692552fd" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 39009 } } peers { permanent_uuid: "24a9bb5a15c64e1e9a0b98a4e019a033" member_type: VOTER last_known_addr { host: "127.11.14.125" port: 37509 } } peers { permanent_uuid: "f2e4bd71d50e45dc85785f371d890706" member_type: VOTER last_known_addr { host: "127.11.14.124" port: 42153 } }
I20260807 18:37:55.306229 3867 sys_catalog.cc:455] T 00000000000000000000000000000000 P 95117623e6424e2f8b2f8beb692552fd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 95117623e6424e2f8b2f8beb692552fd. Latest consensus state: current_term: 1 leader_uuid: "95117623e6424e2f8b2f8beb692552fd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95117623e6424e2f8b2f8beb692552fd" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 39009 } } peers { permanent_uuid: "24a9bb5a15c64e1e9a0b98a4e019a033" member_type: VOTER last_known_addr { host: "127.11.14.125" port: 37509 } } peers { permanent_uuid: "f2e4bd71d50e45dc85785f371d890706" member_type: VOTER last_known_addr { host: "127.11.14.124" port: 42153 } } }
I20260807 18:37:55.306350 3867 sys_catalog.cc:458] T 00000000000000000000000000000000 P 95117623e6424e2f8b2f8beb692552fd [sys.catalog]: This master's current role is: LEADER
I20260807 18:37:55.298144 3615 master_runner.cc:186] Error getting master registration for 127.11.14.124:42153: OK, instance_id { permanent_uuid: "f2e4bd71d50e45dc85785f371d890706" instance_seqno: 1786127875235123 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 18:37:55.307469 3615 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260807 18:37:55.308928 3894 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 24a9bb5a15c64e1e9a0b98a4e019a033: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 18:37:55.309003 3894 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 18:37:55.309314 3899 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:37:55.307493 3859 raft_consensus.cc:348] T 00000000000000000000000000000000 P f2e4bd71d50e45dc85785f371d890706 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95117623e6424e2f8b2f8beb692552fd" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 39009 } } peers { permanent_uuid: "24a9bb5a15c64e1e9a0b98a4e019a033" member_type: VOTER last_known_addr { host: "127.11.14.125" port: 37509 } } peers { permanent_uuid: "f2e4bd71d50e45dc85785f371d890706" member_type: VOTER last_known_addr { host: "127.11.14.124" port: 42153 } }
I20260807 18:37:55.309389 3859 raft_consensus.cc:729] T 00000000000000000000000000000000 P f2e4bd71d50e45dc85785f371d890706 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2e4bd71d50e45dc85785f371d890706, State: Initialized, Role: FOLLOWER
I20260807 18:37:55.309746 3859 consensus_queue.cc:260] T 00000000000000000000000000000000 P f2e4bd71d50e45dc85785f371d890706 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95117623e6424e2f8b2f8beb692552fd" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 39009 } } peers { permanent_uuid: "24a9bb5a15c64e1e9a0b98a4e019a033" member_type: VOTER last_known_addr { host: "127.11.14.125" port: 37509 } } peers { permanent_uuid: "f2e4bd71d50e45dc85785f371d890706" member_type: VOTER last_known_addr { host: "127.11.14.124" port: 42153 } }
I20260807 18:37:55.310467 3899 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:37:55.310371 3905 sys_catalog.cc:455] T 00000000000000000000000000000000 P f2e4bd71d50e45dc85785f371d890706 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95117623e6424e2f8b2f8beb692552fd" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 39009 } } peers { permanent_uuid: "24a9bb5a15c64e1e9a0b98a4e019a033" member_type: VOTER last_known_addr { host: "127.11.14.125" port: 37509 } } peers { permanent_uuid: "f2e4bd71d50e45dc85785f371d890706" member_type: VOTER last_known_addr { host: "127.11.14.124" port: 42153 } } }
I20260807 18:37:55.310748 3905 sys_catalog.cc:458] T 00000000000000000000000000000000 P f2e4bd71d50e45dc85785f371d890706 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 18:37:55.311173 3859 sys_catalog.cc:564] T 00000000000000000000000000000000 P f2e4bd71d50e45dc85785f371d890706 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:37:55.311246 3830 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "95117623e6424e2f8b2f8beb692552fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "f2e4bd71d50e45dc85785f371d890706"
I20260807 18:37:55.311468 3830 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f2e4bd71d50e45dc85785f371d890706 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:37:55.312891 3830 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f2e4bd71d50e45dc85785f371d890706 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95117623e6424e2f8b2f8beb692552fd in term 1.
W20260807 18:37:55.317160 3919 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f2e4bd71d50e45dc85785f371d890706: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 18:37:55.317240 3919 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 18:37:55.328593 3863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:55.314452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46356 (local address 127.11.14.124:42153)
0807 18:37:55.314566 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0807 18:37:55.314571 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:37:55.318495 (+ 3924us) server_negotiation.cc:407] Connection header received
0807 18:37:55.318705 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:37:55.318708 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:37:55.318758 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:37:55.318813 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:37:55.320414 (+ 1601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:55.321179 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:55.322373 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:55.322707 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:55.322785 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:55.324754 (+ 1969us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:37:55.324758 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:37:55.324761 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:37:55.324787 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:37:55.326552 (+ 1765us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:37:55.327083 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:37:55.327088 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:37:55.327090 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:37:55.327177 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:37:55.327513 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:37:55.327517 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:37:55.327519 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:37:55.327781 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:37:55.327915 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0807 18:37:55.328319 (+ 404us) server_negotiation.cc:299] Negotiation successful
0807 18:37:55.328437 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260807 18:37:55.335971 3775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:55.315749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52956 (local address 127.11.14.125:37509)
0807 18:37:55.315858 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0807 18:37:55.315863 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:37:55.321676 (+ 5813us) server_negotiation.cc:407] Connection header received
0807 18:37:55.321784 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:37:55.321787 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:37:55.321833 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:37:55.321911 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:37:55.328358 (+ 6447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:55.329086 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:55.330149 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:55.330432 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:55.330495 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:55.331977 (+ 1482us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:37:55.331981 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:37:55.331984 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:37:55.332011 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:37:55.333970 (+ 1959us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:37:55.334510 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:37:55.334514 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:37:55.334516 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:37:55.334608 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:37:55.334917 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:37:55.334921 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:37:55.334923 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:37:55.335139 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:37:55.335276 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0807 18:37:55.335675 (+ 399us) server_negotiation.cc:299] Negotiation successful
0807 18:37:55.335804 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260807 18:37:55.338446 3744 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 24a9bb5a15c64e1e9a0b98a4e019a033 [term 1 FOLLOWER]: Refusing update from remote peer 95117623e6424e2f8b2f8beb692552fd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:37:55.339201 3867 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 95117623e6424e2f8b2f8beb692552fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "24a9bb5a15c64e1e9a0b98a4e019a033" member_type: VOTER last_known_addr { host: "127.11.14.125" port: 37509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:37:55.339469 3830 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f2e4bd71d50e45dc85785f371d890706 [term 1 FOLLOWER]: Refusing update from remote peer 95117623e6424e2f8b2f8beb692552fd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:37:55.341264 3695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:55.315263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54012 (local address 127.11.14.126:39009)
0807 18:37:55.315380 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0807 18:37:55.315385 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:37:55.319536 (+ 4151us) server_negotiation.cc:407] Connection header received
0807 18:37:55.320455 (+ 919us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:37:55.320459 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:37:55.320513 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:37:55.320566 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:37:55.322855 (+ 2289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:55.323655 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:55.331665 (+ 8010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:55.332043 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:55.332124 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:55.336703 (+ 4579us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:37:55.336708 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:37:55.336711 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:37:55.336744 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:37:55.338803 (+ 2059us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:37:55.339854 (+ 1051us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:37:55.339858 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:37:55.339860 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:37:55.339959 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:37:55.340295 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:37:55.340299 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:37:55.340300 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:37:55.340521 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:37:55.340649 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0807 18:37:55.340992 (+ 343us) server_negotiation.cc:299] Negotiation successful
0807 18:37:55.341110 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260807 18:37:55.343997 3868 sys_catalog.cc:455] T 00000000000000000000000000000000 P 24a9bb5a15c64e1e9a0b98a4e019a033 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 95117623e6424e2f8b2f8beb692552fd. Latest consensus state: current_term: 1 leader_uuid: "95117623e6424e2f8b2f8beb692552fd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95117623e6424e2f8b2f8beb692552fd" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 39009 } } peers { permanent_uuid: "24a9bb5a15c64e1e9a0b98a4e019a033" member_type: VOTER last_known_addr { host: "127.11.14.125" port: 37509 } } peers { permanent_uuid: "f2e4bd71d50e45dc85785f371d890706" member_type: VOTER last_known_addr { host: "127.11.14.124" port: 42153 } } }
I20260807 18:37:55.344125 3868 sys_catalog.cc:458] T 00000000000000000000000000000000 P 24a9bb5a15c64e1e9a0b98a4e019a033 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 18:37:55.344271 3897 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 95117623e6424e2f8b2f8beb692552fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2e4bd71d50e45dc85785f371d890706" member_type: VOTER last_known_addr { host: "127.11.14.124" port: 42153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:37:55.348605 3905 sys_catalog.cc:455] T 00000000000000000000000000000000 P f2e4bd71d50e45dc85785f371d890706 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 95117623e6424e2f8b2f8beb692552fd. Latest consensus state: current_term: 1 leader_uuid: "95117623e6424e2f8b2f8beb692552fd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95117623e6424e2f8b2f8beb692552fd" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 39009 } } peers { permanent_uuid: "24a9bb5a15c64e1e9a0b98a4e019a033" member_type: VOTER last_known_addr { host: "127.11.14.125" port: 37509 } } peers { permanent_uuid: "f2e4bd71d50e45dc85785f371d890706" member_type: VOTER last_known_addr { host: "127.11.14.124" port: 42153 } } }
I20260807 18:37:55.348753 3905 sys_catalog.cc:458] T 00000000000000000000000000000000 P f2e4bd71d50e45dc85785f371d890706 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 18:37:55.349960 3897 sys_catalog.cc:455] T 00000000000000000000000000000000 P 95117623e6424e2f8b2f8beb692552fd [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "95117623e6424e2f8b2f8beb692552fd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95117623e6424e2f8b2f8beb692552fd" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 39009 } } peers { permanent_uuid: "24a9bb5a15c64e1e9a0b98a4e019a033" member_type: VOTER last_known_addr { host: "127.11.14.125" port: 37509 } } peers { permanent_uuid: "f2e4bd71d50e45dc85785f371d890706" member_type: VOTER last_known_addr { host: "127.11.14.124" port: 42153 } } }
I20260807 18:37:55.350085 3897 sys_catalog.cc:458] T 00000000000000000000000000000000 P 95117623e6424e2f8b2f8beb692552fd [sys.catalog]: This master's current role is: LEADER
I20260807 18:37:55.351470 3909 mvcc.cc:205] Tried to move back new op lower bound from 7315979777371815936 to 7315979777250541568. Current Snapshot: MvccSnapshot[applied={T|T < 7315979777371815936}]
I20260807 18:37:55.351697 3868 sys_catalog.cc:455] T 00000000000000000000000000000000 P 24a9bb5a15c64e1e9a0b98a4e019a033 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "95117623e6424e2f8b2f8beb692552fd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95117623e6424e2f8b2f8beb692552fd" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 39009 } } peers { permanent_uuid: "24a9bb5a15c64e1e9a0b98a4e019a033" member_type: VOTER last_known_addr { host: "127.11.14.125" port: 37509 } } peers { permanent_uuid: "f2e4bd71d50e45dc85785f371d890706" member_type: VOTER last_known_addr { host: "127.11.14.124" port: 42153 } } }
I20260807 18:37:55.351804 3868 sys_catalog.cc:458] T 00000000000000000000000000000000 P 24a9bb5a15c64e1e9a0b98a4e019a033 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 18:37:55.353778 3867 sys_catalog.cc:455] T 00000000000000000000000000000000 P 95117623e6424e2f8b2f8beb692552fd [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "95117623e6424e2f8b2f8beb692552fd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95117623e6424e2f8b2f8beb692552fd" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 39009 } } peers { permanent_uuid: "24a9bb5a15c64e1e9a0b98a4e019a033" member_type: VOTER last_known_addr { host: "127.11.14.125" port: 37509 } } peers { permanent_uuid: "f2e4bd71d50e45dc85785f371d890706" member_type: VOTER last_known_addr { host: "127.11.14.124" port: 42153 } } }
I20260807 18:37:55.353911 3867 sys_catalog.cc:458] T 00000000000000000000000000000000 P 95117623e6424e2f8b2f8beb692552fd [sys.catalog]: This master's current role is: LEADER
I20260807 18:37:55.354692 3905 sys_catalog.cc:455] T 00000000000000000000000000000000 P f2e4bd71d50e45dc85785f371d890706 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "95117623e6424e2f8b2f8beb692552fd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95117623e6424e2f8b2f8beb692552fd" member_type: VOTER last_known_addr { host: "127.11.14.126" port: 39009 } } peers { permanent_uuid: "24a9bb5a15c64e1e9a0b98a4e019a033" member_type: VOTER last_known_addr { host: "127.11.14.125" port: 37509 } } peers { permanent_uuid: "f2e4bd71d50e45dc85785f371d890706" member_type: VOTER last_known_addr { host: "127.11.14.124" port: 42153 } } }
I20260807 18:37:55.354815 3905 sys_catalog.cc:458] T 00000000000000000000000000000000 P f2e4bd71d50e45dc85785f371d890706 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 18:37:55.359369 3899 catalog_manager.cc:1347] Generated new cluster ID: 3ec6def3a40740678cac532e04701b84
I20260807 18:37:55.359439 3899 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:37:55.368952 3927 mvcc.cc:205] Tried to move back new op lower bound from 7315979777371815936 to 7315979777250541568. Current Snapshot: MvccSnapshot[applied={T|T < 7315979777371815936}]
I20260807 18:37:55.370775 3924 mvcc.cc:205] Tried to move back new op lower bound from 7315979777371815936 to 7315979777250541568. Current Snapshot: MvccSnapshot[applied={T|T < 7315979777371815936}]
I20260807 18:37:55.382511 3899 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:37:55.383488 3899 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:37:55.403824 3899 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 95117623e6424e2f8b2f8beb692552fd: Generated new TSK 0
I20260807 18:37:55.404788 3899 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 18:37:55.439247 3775 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:37:55.437203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52966 (local address 127.11.14.125:37509)
0807 18:37:55.437319 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0807 18:37:55.437324 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:37:55.438174 (+ 850us) server_negotiation.cc:407] Connection header received
0807 18:37:55.438370 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:37:55.438374 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:37:55.438422 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:37:55.438476 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:37:55.439109 (+ 633us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52966: BlockingRecv error: recv got EOF from 127.0.0.1:52966 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
W20260807 18:37:55.441146 3863 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:37:55.439932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46370 (local address 127.11.14.124:42153)
0807 18:37:55.440041 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0807 18:37:55.440046 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:37:55.440267 (+ 221us) server_negotiation.cc:407] Connection header received
0807 18:37:55.440433 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:37:55.440436 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:37:55.440482 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:37:55.440532 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:37:55.441021 (+ 489us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46370: BlockingRecv error: recv got EOF from 127.0.0.1:46370 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
W20260807 18:37:55.442086 3695 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:37:55.436559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54020 (local address 127.11.14.126:39009)
0807 18:37:55.436735 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0807 18:37:55.436741 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:37:55.437869 (+ 1128us) server_negotiation.cc:407] Connection header received
0807 18:37:55.438012 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:37:55.438016 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:37:55.438071 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:37:55.438164 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:37:55.441968 (+ 3804us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54020: BlockingRecv error: recv got EOF from 127.0.0.1:54020 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
W20260807 18:37:55.446385 3775 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:37:55.444870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52978 (local address 127.11.14.125:37509)
0807 18:37:55.444967 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0807 18:37:55.444971 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:37:55.445453 (+ 482us) server_negotiation.cc:407] Connection header received
0807 18:37:55.445662 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:37:55.445665 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:37:55.445753 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:37:55.445805 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:37:55.446287 (+ 482us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52978: BlockingRecv error: recv got EOF from 127.0.0.1:52978 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
W20260807 18:37:55.451301 3775 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:37:55.449762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52994 (local address 127.11.14.125:37509)
0807 18:37:55.449858 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0807 18:37:55.449862 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:37:55.450451 (+ 589us) server_negotiation.cc:407] Connection header received
0807 18:37:55.450632 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:37:55.450635 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:37:55.450684 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:37:55.450745 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:37:55.451209 (+ 464us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52994: BlockingRecv error: recv got EOF from 127.0.0.1:52994 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
W20260807 18:37:55.456736 3775 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:37:55.455250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52996 (local address 127.11.14.125:37509)
0807 18:37:55.455344 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0807 18:37:55.455347 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 18:37:55.455887 (+ 540us) server_negotiation.cc:407] Connection header received
0807 18:37:55.456066 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:37:55.456069 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:37:55.456116 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:37:55.456166 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:37:55.456629 (+ 463us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52996: BlockingRecv error: recv got EOF from 127.0.0.1:52996 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
W20260807 18:37:55.464267 3775 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:37:55.462579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53004 (local address 127.11.14.125:37509)
0807 18:37:55.462755 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0807 18:37:55.462760 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:37:55.463301 (+ 541us) server_negotiation.cc:407] Connection header received
0807 18:37:55.463484 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:37:55.463487 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:37:55.463534 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:37:55.463678 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:37:55.464174 (+ 496us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53004: BlockingRecv error: recv got EOF from 127.0.0.1:53004 (error 108)
Metrics: {"server-negotiator.queue_time_us":87}
W20260807 18:37:55.476085 3775 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:37:55.474351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53012 (local address 127.11.14.125:37509)
0807 18:37:55.474459 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0807 18:37:55.474463 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:37:55.475035 (+ 572us) server_negotiation.cc:407] Connection header received
0807 18:37:55.475231 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:37:55.475235 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:37:55.475288 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:37:55.475340 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:37:55.475972 (+ 632us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53012: BlockingRecv error: recv got EOF from 127.0.0.1:53012 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
W20260807 18:37:55.497987 3775 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:37:55.496156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53024 (local address 127.11.14.125:37509)
0807 18:37:55.496286 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0807 18:37:55.496292 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:37:55.496916 (+ 624us) server_negotiation.cc:407] Connection header received
0807 18:37:55.497143 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:37:55.497146 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:37:55.497201 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:37:55.497260 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:37:55.497860 (+ 600us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53024: BlockingRecv error: recv got EOF from 127.0.0.1:53024 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
W20260807 18:37:55.534619 3775 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:37:55.533027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53034 (local address 127.11.14.125:37509)
0807 18:37:55.533183 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0807 18:37:55.533189 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:37:55.533505 (+ 316us) server_negotiation.cc:407] Connection header received
0807 18:37:55.533679 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:37:55.533683 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:37:55.533755 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:37:55.533863 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:37:55.534493 (+ 630us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53034: BlockingRecv error: recv got EOF from 127.0.0.1:53034 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
W20260807 18:37:55.601953 3775 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:37:55.601041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53044 (local address 127.11.14.125:37509)
0807 18:37:55.601186 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0807 18:37:55.601190 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:37:55.601207 (+ 17us) server_negotiation.cc:407] Connection header received
0807 18:37:55.601301 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:37:55.601304 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:37:55.601352 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:37:55.601437 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:37:55.601852 (+ 415us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53044: BlockingRecv error: recv got EOF from 127.0.0.1:53044 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
W20260807 18:37:55.733640 3775 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:37:55.732566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53050 (local address 127.11.14.125:37509)
0807 18:37:55.732710 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0807 18:37:55.732714 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:37:55.732880 (+ 166us) server_negotiation.cc:407] Connection header received
0807 18:37:55.732995 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:37:55.732998 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:37:55.733055 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:37:55.733141 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:37:55.733534 (+ 393us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53050: BlockingRecv error: recv got EOF from 127.0.0.1:53050 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
W20260807 18:37:55.993577 3775 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:37:55.992361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53060 (local address 127.11.14.125:37509)
0807 18:37:55.992512 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0807 18:37:55.992516 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:37:55.992649 (+ 133us) server_negotiation.cc:407] Connection header received
0807 18:37:55.992811 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:37:55.992813 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:37:55.992862 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:37:55.992981 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:37:55.993445 (+ 464us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53060: BlockingRecv error: recv got EOF from 127.0.0.1:53060 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
I20260807 18:37:56.310451 3894 catalog_manager.cc:1259] Loaded cluster ID: 3ec6def3a40740678cac532e04701b84
I20260807 18:37:56.310583 3894 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 24a9bb5a15c64e1e9a0b98a4e019a033: loading cluster ID for follower catalog manager: success
I20260807 18:37:56.312588 3894 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 24a9bb5a15c64e1e9a0b98a4e019a033: acquiring CA information for follower catalog manager: success
I20260807 18:37:56.313886 3894 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 24a9bb5a15c64e1e9a0b98a4e019a033: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 18:37:56.318472 3919 catalog_manager.cc:1259] Loaded cluster ID: 3ec6def3a40740678cac532e04701b84
I20260807 18:37:56.318559 3919 catalog_manager.cc:1552] T 00000000000000000000000000000000 P f2e4bd71d50e45dc85785f371d890706: loading cluster ID for follower catalog manager: success
I20260807 18:37:56.320287 3919 catalog_manager.cc:1574] T 00000000000000000000000000000000 P f2e4bd71d50e45dc85785f371d890706: acquiring CA information for follower catalog manager: success
I20260807 18:37:56.321368 3919 catalog_manager.cc:1602] T 00000000000000000000000000000000 P f2e4bd71d50e45dc85785f371d890706: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 18:37:56.512204 4011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:56.507910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53064 (local address 127.11.14.125:37509)
0807 18:37:56.508154 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0807 18:37:56.508158 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:37:56.508287 (+ 129us) server_negotiation.cc:407] Connection header received
0807 18:37:56.508476 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:37:56.508479 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:37:56.508610 (+ 131us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:37:56.508796 (+ 186us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:37:56.509065 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:56.509595 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:56.510489 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:56.510732 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:56.510846 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:56.511248 (+ 402us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:37:56.511750 (+ 502us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:37:56.511896 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0807 18:37:56.511942 (+ 46us) server_negotiation.cc:299] Negotiation successful
0807 18:37:56.512000 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":67,"threads_started":1}
I20260807 18:37:56.519346 4019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:56.515424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46372 (local address 127.11.14.124:42153)
0807 18:37:56.515746 (+ 322us) server_negotiation.cc:206] Beginning negotiation
0807 18:37:56.515750 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:37:56.515763 (+ 13us) server_negotiation.cc:407] Connection header received
0807 18:37:56.515848 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:37:56.515850 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:37:56.515968 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:37:56.516070 (+ 102us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:37:56.516320 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:56.516858 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:56.517716 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:56.518018 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:56.518128 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:56.518477 (+ 349us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:37:56.518907 (+ 430us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:37:56.519054 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0807 18:37:56.519115 (+ 61us) server_negotiation.cc:299] Negotiation successful
0807 18:37:56.519174 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":248,"thread_start_us":82,"threads_started":1}
I20260807 18:37:56.526121 4019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:56.522371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46376 (local address 127.11.14.124:42153)
0807 18:37:56.522504 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0807 18:37:56.522508 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:37:56.522523 (+ 15us) server_negotiation.cc:407] Connection header received
0807 18:37:56.522687 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:37:56.522693 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:37:56.522751 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:37:56.523110 (+ 359us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:37:56.523216 (+ 106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:56.524005 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:56.525065 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:56.525297 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:56.525406 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:56.525688 (+ 282us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:37:56.525765 (+ 77us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:37:56.525833 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0807 18:37:56.525919 (+ 86us) server_negotiation.cc:299] Negotiation successful
0807 18:37:56.525970 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260807 18:37:56.526975 4011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:56.522698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53074 (local address 127.11.14.125:37509)
0807 18:37:56.522831 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0807 18:37:56.522835 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:37:56.522971 (+ 136us) server_negotiation.cc:407] Connection header received
0807 18:37:56.523079 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:37:56.523082 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:37:56.523131 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:37:56.523204 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:37:56.523471 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:56.524215 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:56.525632 (+ 1417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:56.525967 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:56.526063 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:56.526454 (+ 391us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:37:56.526561 (+ 107us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:37:56.526611 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0807 18:37:56.526801 (+ 190us) server_negotiation.cc:299] Negotiation successful
0807 18:37:56.526859 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260807 18:37:56.527688 4029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:37:56.522467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54034 (local address 127.11.14.126:39009)
0807 18:37:56.522989 (+ 522us) server_negotiation.cc:206] Beginning negotiation
0807 18:37:56.522995 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:37:56.523008 (+ 13us) server_negotiation.cc:407] Connection header received
0807 18:37:56.523099 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:37:56.523102 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:37:56.523259 (+ 157us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:37:56.523372 (+ 113us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:37:56.523649 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:56.524242 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:56.526131 (+ 1889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:37:56.526446 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:37:56.526551 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:37:56.526904 (+ 353us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:37:56.527258 (+ 354us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:37:56.527357 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0807 18:37:56.527474 (+ 117us) server_negotiation.cc:299] Negotiation successful
0807 18:37:56.527521 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":441,"thread_start_us":86,"threads_started":1}
I20260807 18:37:56.537561 11321 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkz56XN/build/debug/bin/kudu with pid 3615
I20260807 18:37:56.548044 11321 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkz56XN/build/debug/bin/kudu with pid 3698
I20260807 18:37:56.555799 11321 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskkz56XN/build/debug/bin/kudu with pid 3781
2026-08-07T18:37:56Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (3214 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (6499 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (296622 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST